Multi-agent systems learn when to collaborate for better code results
Learning How Much to Collaborate: Difficulty-Aware Topology Selection for Multi-Agent Code Generation
Multiagent SystemsSoftware Engineering
Summary
Different ways for smart programs to work together can help them write better code, especially on harder problems. The authors found that teaming up in a complex hierarchy works much better for difficult tasks but costs more effort. They created a system that predicts which way of teaming up is best for each problem, balancing success with effort. This system chooses smarter collaborations, improving coding results across many tests, including math problems.
What this means in practice
- •For software engineering teams: Improve automated code generation tools by selecting collaboration styles based on problem difficulty to boost success rates efficiently.
- •For automated reasoning developers: Enhance mathematical reasoning systems by adapting collaboration structures in multi-agent AI models to solve tougher problems better.
Authors
Yunsong Hong
Abstract
Multi-agent systems for code generation are deployed with a single communication topology, chosen once for every problem. This is the wrong granularity. Evaluating five topologies on 614 problems from APPS, HumanEval+ and LiveCodeBench, we find that the advantage of hierarchical collaboration over a single agent grows from 2.4 points of pass@1 on the easiest third of problems to 21.1 points on the hardest third, while its token cost stays about ten times higher. We propose the Difficulty-Aware Topology Selector (DATS), which predicts each topology's probability of solving a problem and selects the one maximising predicted success minus cost. Its predictor is a graph network that treats the five topologies as nodes of a connectivity order rather than independent labels, worth 1.7 points over a flat multi-label head. Because the cost penalty is a single scalar recalibrable without retraining, routers compare at equal spend: under this budget-matched protocol six cost-aware methods span 21.6 percentage points, and two baselines leading DATS fall behind once calibrated to it. Fixed at 40% of the always-hierarchical cost, DATS reaches 77.7% pass@1 against 73.6% (always-hierarchical) and 74.3% (strongest learned competitor), all eleven pairwise McNemar comparisons surviving Holm-Bonferroni correction. The 4.1-point gain holds across four backbones spanning fourteen points of capability, and replacing the 39 interpretable features with a graph network or a pretrained encoder shifts accuracy by at most 1.3 points, never significantly. A cross-domain study on 400 mathematical reasoning problems reproduces the effect, the gap widening from 2.5 to 20.9 points.