Multi-agent vision language models coordinate better with learned messaging
CoLA: Multi-VLA Coordination via Learned Adapters
RoboticsMachine LearningMultiagent Systems
Summary
Sometimes, multiple agents or robots working together need to share information to complete tasks efficiently. The authors show that when agents can secretly see or know something, they can improve teamwork by sending learned messages to each other. Their method, CoLA, adds a communication channel on top of existing models, which helps especially when some information is hidden from certain agents. This helps improve success in tasks like handing over objects where one agent might not see everything.
What this means in practice
- •For robotics engineers: Build robots that share limited or private observations via learned message channels for more efficient collaborative tasks in shared workspaces.
- •For manufacturing automation teams: Improve coordination of multi-arm systems handling partially occluded objects by integrating learned communication adapters with existing vision-language-action models.
Authors
Ahaskar Kashyap, Jiawen Zong, Boyu Ma, Chuhao Zhou, Jingliang Li, Stefano V. Albrecht, Jianfei Yang
Abstract
Vision-Language-Action (VLA) models are typically deployed as isolated single-agent policies. When several such agents share a workspace, they can coordinate implicitly, by observing one another, or explicitly, by exchanging messages, but it remains unclear when explicit communication becomes necessary. We introduce CoLA (Coordination via Learned Adapters), which keeps each agent's VLA backbone frozen and trains a lightweight message channel and a per-arm diffusion action head on top, with fully decentralised execution. Across three tasks that differ in how much each agent can observe, the channel matters only when information is private. On an occluded tray handover, where the box's colour marker is visible to one agent only, CoLA reaches 76.0% success versus 1.3% with messages zeroed throughout training. On two-arm and three-arm handover tasks, removing messages has no measurable effect with a shared overhead camera (90.6% vs. 88.3% at two arms; 63.5% vs. 60.0% at three), but lowers success on the two-arm task's wrist-only variant (74.8% vs. 64.3%). This architecture generalises across different VLA backbones as well. With a frozen $π_{0.5}$ backbone on the same occluded tray task, CoLA matches the overall success rate of a LoRA-fine-tuned baseline (37.3% vs. 34.7%) using a fraction of the trainable parameters. When CoLA places the box, it selects the correct tray 71.8% of the time, whereas the independently fine-tuned baseline, whose placing arm never sees the marker, selects it at chance level (34.7% against 33.3%).