System adapts collaboration by measuring ai competence on tasks
Competence-Aware Orchestration for LLM-Based Multi-Agent Systems
Artificial Intelligence
Summary
Different AI tools have strengths that vary depending on the question being asked. The authors introduce a way to figure out how good an AI is at a task before deciding how many helpers it should use and how they talk to each other. This helps make teamwork smarter, spending less effort when the AI is strong and adding more support when it's weaker. Their tests showed this approach improves accuracy while saving resources across many AI systems and tasks.
What this means in practice
- •For ai system builders: Set up multi-agent AI systems that dynamically adjust teamwork to save costs and improve accuracy based on the AI's skill in real time.
- •For enterprise automation teams: Control AI workflows that allocate varying collaboration resources depending on task difficulty and AI capability to optimize efficiency.
Authors
Songmiao Wang, Yuchen Wang, Chenglin Yu, Hongxia Yang, Ming Li
Abstract
LLM-based multi-agent systems (MAS) have emerged as a promising paradigm for enhancing complex reasoning through role specialization and collaboration, yet a key challenge remains: how can collaboration be adapted to a backbone's competence on the current query? This problem matters because more collaboration is not always better--richer coordination may help on difficult cases, but it also incurs additional token cost and redundant interactions. We propose a two-stage competence-aware framework for automatic MAS orchestration. It first models the LLM backbone's task-conditioned competence for the current query, and then uses this signal to adapt role activation and the communication graph. Across six benchmarks, our method achieves an average score of 93.81%. Experiments with six LLM backbones of varying competence levels further show that our method adjusts collaboration according to the inferred competence: for higher-competence backbones, it improves average accuracy from 87.22 to 89.97 while reducing token cost by 36.26%; for lower-competence backbones, it yields a further average gain of 4.58 points over existing methods. The same trend also holds on unseen backbones.