Decentralized load sharing boosts multi-agent AI task handling
DEALS: Decentralized Expertise-Aware Load Serving for Multi-Agent LLM Systems
Machine LearningArtificial IntelligenceMultiagent Systems
Summary
When many AI agents work together to solve complex problems, it's hard to organize who does what without slowing things down or costing too much. The authors propose DEALS, a way for these agents to share tasks and help each other without a central boss controlling everything. Each agent decides if it should do a task or pass it along, based on how busy it is and how well it can handle the job. This approach makes the whole system faster and better at solving problems by balancing work and expertise across agents.
What this means in practice
- •For ai system engineers: Manage complex AI workflows by enabling agents to self-route tasks efficiently without central control.
- •For cloud service operators: Improve utilization and throughput of distributed AI resources by balancing workloads based on agent expertise and queue backlogs.
Authors
Jingjuan Huang, Wenbin Wang, Yanchuan Yin, Alvaro Velasquez, Jia Liu
Abstract
Multi-agent systems (MAS) have recently emerged as an effective approach for coordinating large language model (LLM)-based agents to solve complex tasks through structured interactions. In practice, MASs often handle a stream of heterogeneous and complex tasks, requiring agents to decompose each task and then self-organize and self-evolve to adapt to incoming tasks while sharing execution resources. However, most early approaches to MASs rely on centralized controllers or fixed coordination patterns, which can limit scalability or adaptability. In contrast, existing decentralized and dynamic MASs often require training dedicated routers or invoking LLMs for agent selection, resulting in substantial computational costs and coordination overhead. To address these challenges and enable efficient task-level self-organization and self-evolution for task- and workload-level collaboration, we propose Decentralized Expertise-Aware Load Serving (DEALS), a decentralized and low-complexity framework that enables agents to self-organize and dynamically route concurrent tasks for processing. Specifically, each agent maintains local queues of incoming tasks, and its router decides whether to process a task locally or forward it to a neighbor based on differences in backlog and success rate. Meanwhile, executors process independent tasks concurrently within and across agents, and partially solved tasks can be resumed by other agents. Experiments show that DEALS not only improves performance along multiple dimensions (e.g., answer accuracy and task throughput) in both homogeneous and heterogeneous agent pools, but also balances agent expertise and workload in a self-organized manner, enabling effective decentralized coordination.