Coordinating diverse AI models for efficient joint task solving
Toward Collective Model Intelligence: Characterizing and Coordinating Models
Artificial Intelligence
Summary
As more AI models are created independently, they each have their own strengths and weaknesses. The paper presents ways to understand what each model can do without testing everything, and how to pick and combine models to work together on different tasks efficiently. They also built a system to handle running these models in a smart way to save time and resources. This approach showed it could better estimate model skills and do tasks more cheaply and quickly than other methods they compared it to.
What this means in practice
- •For application developers: Combine multiple AI models dynamically to meet changing task needs efficiently with less cost and delay.
- •For cloud infrastructure teams: Manage deployment and scheduling of diverse AI models to optimize resource use and speed up response times.
Authors
SinapisAI Team, Jian-Dong Liu, Qun Dai, Zhendong Du, Guannan Lai, Hao-Yu Liu, Yi-Fan Ma, Zi-Yu Mao, Tengzhong Sun, Xin-Yi Zhang, Zi-Chen Zhao, Gelin Bian, Zongyuan Cai, Ju Cang, Zhangqin Chen, Shilei Gu, Guoding Ji, Taojian Jiang, Hao-Xuan Ma, Fan Wu, Lin-Xing Wu, Tao Yu, Xiao Zhang, Tianyi Zhou, Rong Gu, Ming Li, Zhi-Hao Tan, Han-Jia Ye, Yang Yu, De-Chuan Zhan
Abstract
Independently developed AI models accumulate diverse and complementary capabilities, but a growing model collection does not automatically realize Collective AI. Addressing this gap requires understanding model capabilities with limited evaluation, organizing complementary models to collaborate on diverse tasks, and executing model calls efficiently under resource constraints. We present a systematic approach to pursuing Collective Model Intelligence through three connected components. First, Model Eval constructs evidence-based capability profiles by combining query-level performance estimation with cross-domain prediction, reducing the need for exhaustive evaluation. Second, Mio (Models in One) organizes complementary model candidates and supports task-conditioned, cost-aware selection and collaboration as task requirements evolve. Third, Mos (Model Serving OS) separates model selection from execution and provides endpoint scheduling and shared model deployment and distribution. Experiments show lower capability-estimation errors under limited evaluation. Across nine public benchmarks, Mio 1.0 Lite achieves the highest average score and lowest average cost among compared methods, with per-request costs 73.7-97.5% lower than those of the best-scoring alternatives. A prewarmed shared cache accelerates model-file retrieval by up to 23.3 times. This work lays a foundation for Collective Model Intelligence by characterizing and coordinating the diverse capabilities of an expanding model universe.