Distilling Collaborative Dynamics into Latent Space for Implicit Coordination in Decentralized Multi-Agent Manipulation
2026-06-22 • Robotics
Robotics
AI summaryⓘ
The authors designed a new way for multiple robot arms to work together without needing to see everything or talk directly to each other. Their method, called CLS-DP, teaches each robot to understand teamwork from limited local views and shared instructions by learning a hidden collaborative signal. This helps the robots coordinate smoothly even as more join the team, and it works better than older methods on standard robot tasks. The authors also show that each robot pays attention to both its own and its teammates' important parts, which helps them act together implicitly.
multi-agent systemsdecentralized controlpartial observabilitylatent spacecentralized training decentralized execution (CTDE)diffusion denoisingrobot manipulationcollaborative dynamicsRoboFactory benchmark
Authors
Chanyoung Park, Minsung Yoon, Andrew Jeong, Sung-eui Yoon
Abstract
Multi-arm manipulation demands precise spatiotemporal coordination, yet many centralized approaches scale poorly as team size increases. To address this, we propose CLS-DP, a decentralized multi-agent framework that enables implicit coordination under partial observability without shared global views, explicit state information, or inter-agent communication. Under the centralized training and decentralized execution (CTDE) paradigm, CLS-DP distills privileged multi-agent dynamics into a latent space. At deployment, each agent infers a collaborative latent from its local RGB observation and a shared task instruction; it then conditions the diffusion denoising process on this latent. This design enables implicit coordination with a per-agent cost independent of team size. Across six RoboFactory benchmark tasks spanning two to four agents, CLS-DP achieves a 38% mean success rate, outperforming the best centralized baseline (20%) and a decentralized ablation without the collaborative latent (9%). It also maintains superior parameter efficiency across all agent configurations. Attribution maps show that an agent conditioned on the collaborative latent places high attribution on the joints and grippers of both itself and its teammates throughout execution. This suggests that the learned latent efficiently encodes collaborative dynamics from local observation, which facilitates implicit coordination in realistic settings characterized by partial observability.