Reinforcement learning builds efficient mixed-size agent teams for tasks
MAGIC: Mixed-Granularity Agent Graphs via Incremental Construction with Dense-Reward Reinforcement Learning
Machine Learning
Summary
Choosing the right size and setup for groups of AI agents to work together is tricky because different parts of a task need different kinds of teamwork. The authors created MAGIC, a method that builds these groups step-by-step, sometimes as one agent and sometimes as a group, deciding how they link together. To make learning easier, MAGIC gives frequent feedback during this building process, not just at the end. This approach helps build collaboration networks that work better and are faster to run than previous methods.
What this means in practice
- •For software development teams: Create adaptable AI agent teams that mix single agents and groups to optimize task collaboration and execution costs.
- •For automation engineers: Design task-specific multi-agent collaboration networks that efficiently handle subtasks with varying complexity and agent roles.
Authors
Kairui Yang, Ziheng Yi, Xunkai Li, Minghao An, Zhanke Liu, Zekai Chen, Rong-Hua Li
Abstract
Collaboration topology shapes both the performance and execution cost of LLM-based multi-agent systems. Because tasks differ in complexity and required capabilities, recent approaches generate task-specific collaboration graphs that specify agent participation and information flow. However, representative topology generators use either individual agents or predefined groups throughout an organization, overlooking differing collaboration needs across subtasks. Our key insight is to select granularity locally for each functional role, combining fine-grained control with reusable collaboration patterns within one organization. Learning such organizations requires exploring a combinatorial construction space with limited intermediate feedback from final-answer rewards. Therefore, we propose MAGIC, a dense-reward reinforcement learning framework for mixed-granularity graph generation. Specifically, MAGIC constructs a mixed-granularity agent graph by sequentially selecting a functional role, instantiating it as a single agent or reusable group, and connecting it to existing units. We directly optimize the construction policy using returns from trajectories sampled under the current policy and use potential-based reward shaping to provide intermediate feedback from probe-based utility and structural signals while preserving the cumulative task reward. MAGIC outperforms state-of-the-art baselines across eight benchmarks and demonstrates strong inference efficiency in our efficiency study.