Transformer policies can mask robot teamwork problems with same actions
Permutation Robustness Is Not Enough: Action Collapse in Multi-Agent Transformer Policies
RoboticsMachine LearningMultiagent Systems
Summary
When multiple robots work together, their behavior should stay consistent even if you change the order you look at them. The authors found that some AI methods for coordinating robot teams can seem consistent only because all robots pick the same action, which isn’t helpful teamwork. They tested different ways to encourage varied actions across the robots while keeping this consistency. Their findings suggest it’s important to check not just if the combined actions look consistent, but also if the robots keep distinct roles and behaviors.
What this means in practice
- •For robotics teams: Improve multi-robot coordination by ensuring policies maintain diverse actions while staying robust to agent order changes in team decision models.
- •For distributed control engineers: Design better algorithms for controlling groups of agents that avoid collapse into identical actions despite input order permutations in transformer-based models.
Authors
Amit Thakur, Mukesh Singhal
Abstract
Transformer policies are attractive for multi-agent robot learning because self-attention can model interactions among agents. However, multi-agent teams are unordered, while transformers typically process agents as ordered token sequences. We study how this mismatch affects cooperative navigation policies under agent-order permutations. Our results show that low permutation error alone can be misleading: policies may appear robust simply because all agents choose the same action. We therefore evaluate policies using both permutation-consistency metrics and action-collapse diagnostics, including action diversity, same-action fraction, and maximum action frequency. A PPO-ID baseline yields non-collapsed behavior but remains order-sensitive, while strong equivariance regularization can still induce homogeneous behavior. A weak equivariance penalty improves the robustness while preserving more diverse actions for teams with \(N=3\) agents, whereas teams with \(N=4\) agents require substantially smaller regularization weights. These findings suggest that multi-agent transformer policies should be evaluated not only by return and permutation robustness, but also by whether they maintain non-collapsed, differentiated multi-agent behavior.