Multi-agent systems improve by fixing failures during training
MAS-Loop: Beyond One-Shot Orchestration towards Self-Evolving Multi-Agent Systems
Multiagent SystemsArtificial Intelligence
Summary
Many computer programs that involve multiple agents working together are trained to perform tasks by seeing only their final results. The authors found that this misses important clues about mistakes made along the way that could help improve the system. They created MAS-Loop, which records step-by-step actions and uses this information to identify and fix specific problems before trying again. This process helps the system get better over time by learning from failures and successful fixes, leading to stronger overall performance.
What this means in practice
- •For multi-agent system developers: Improve coordination policies by using execution trajectories to detect and fix agent failures during training iterations.
- •For robotics engineers: Enhance multi-robot task execution by refining agent behaviors based on detailed failure analysis from task runs.
Authors
Hehai Lin, Qi Yang, Chengwei Qin
Abstract
Training-time multi-agent system (MAS) design seeks to learn reusable orchestration policies, yet prevailing approaches remain largely one-shot: an Orchestrator generates a system but only observes and learns from its terminal outcome. This paradigm discards execution trajectories that expose system-level failures and contain fine-grained information beyond the terminal outcome, leaving no direct basis for revising the source MAS, evaluating the updated MAS, or learning from the resulting contrast. To this end, we introduce MAS-Loop, an execution-grounded framework that closes this feedback loop by coupling an Orchestrator with a trajectory-conditioned Refiner. For each task, the Orchestrator constructs and executes a source MAS to obtain an outcome score and an intermediate trajectory. Conditioned on the trajectory, the Refiner localizes an actionable failure, formulates a trajectory-grounded diagnosis, and proposes a bounded patch to the agents, information flow, or execution configuration. The executable updated MAS is re-executed on the same task to measure its realized outcome under stochastic execution. This pair-MAS comparison provides complementary learning signals: the Orchestrator learns to generate strong systems with fewer recoverable failures, whereas the Refiner learns to expose and repair observed weaknesses, thereby driving policy-level co-evolution across training iterations. Experiments show that MAS-Loop achieves the strongest overall performance among the evaluated baselines, with consistent gains from refinement across all evaluated benchmarks. The co-evolution process also yields MAS-JudgeCorpus, a dataset of execution-grounded source-updated pairs with orchestration- and trajectory-level supervision, which can provide the community with support for MAS reward modeling and trajectory-aware investigation.