Test-time gradient guidance improves multi-agent cooperation performance
G2MAF: Test-Time Gradient Guidance for Multi-Agent Flow Policies
Artificial Intelligence
Summary
Sometimes, when many agents work together using a fixed plan made from past examples, they pick actions that may not be the best at that moment. The authors propose a way to adjust their decisions a little bit during use, guided by a gradient that helps all agents improve their choices together. This method, called G2MAF, makes teamwork better by nudging the agents toward smarter actions while staying close to their original plan. Their tests showed it boosted performance notably in various multi-agent scenarios with only a small increase in computation time.
What this means in practice
- •For robotics teams: Enhance team coordination in robots by adjusting planned actions in real time for better group performance.
- •For autonomous vehicle operators: Improve coordinated decision-making for fleets of autonomous vehicles by refining joint policies during operation without retraining.
Authors
Guowei Zou, Haitao Wang, Guoxin Wang, Zhiquan Chen, Beiwen Zhang, Guojie Wang, Hejun Wu
Abstract
Offline multi-agent reinforcement learning (MARL) learns cooperative policies from fixed datasets without further environment interaction and a learned policy is frozen at deployment. Such a frozen policy typically proposes a single joint action and executes it directly at deployment time. However, this one-shot deployment often commits to a suboptimal proposal, even when better nearby alternatives remain consistent with the behavior data. To address this issue, we propose Gradient Guided Multi Agent Flow (G2MAF), a refinement framework for optimizing joint policies at test-time. G2MAF applies one globally normalized, projected critic gradient to guide and coordinate all agents' corrections while keeping the action both feasible and close to the frozen policy proposal. Across 24 MPE and SMAC settings, its canonical variant improves 20 frozen settings, with mean relative gains of 9.2% on MPE and 8.9% on SMAC, with model inference latency increased by about 6% only.