Multi-operator virtual platform enables humanoid robot teamwork data
MATE: Multi-Agent Virtual Teleoperation Platform for Humanoid Collaboration Data Collection
Robotics
Summary
Humanoid robots need lots of practice working together on tasks, but using real robots for this is expensive and complicated. The authors created MATE, a virtual platform that lets several people in different places control robot helpers at the same time in the same virtual space. This allows collecting lots of teamwork data without needing physical robots or special spaces. They also designed a method called EAIS to better learn from these teamwork examples. Their tests show these virtual lessons can help robots learn to work together and even perform well on real robots without extra training.
What this means in practice
- •For robotics development teams: Collect multi-robot collaboration data remotely using virtual humanoid control to train complex teamwork behaviors without physical hardware.
- •For robotics policy developers: Test and evaluate multi-agent collaborative behaviors in simulation before deploying physical humanoid robots in real-world teamwork scenarios.
Authors
Yichuan Yu, Youzhuo Wang, Yiming Ren, Di Feng, Yexuan Yang, Bingxi Yang, Shengxiao Gong, Yujing Sun, Yuexin Ma
Abstract
Humanoid robots require diverse embodied experiences to acquire complex loco-manipulation and collaborative skills. However, existing humanoid data pipelines primarily focus on individual agents, while physical multi-robot collaboration remains difficult to scale due to costly hardware, dedicated spaces, and repeated resets. In this work, we introduce MATE, a Multi-Agent virtual TEleoperation platform for humanoid collaboration data collection that enables multiple geographically distributed operators to simultaneously control whole-body humanoids in a shared physics-based environment. MATE removes the need for multiple physical robots and co-located operation while preserving physically coupled interactions among humanoids, objects, and environments. Using MATE, we construct a multi-humanoid collaboration dataset comprising 24.1 hours of coordinated behavior across 2,500 joint episodes and five long-horizon tasks, including object handover, relay delivery, environment interaction, and cooperative transport. To improve learning from these interaction-rich demonstrations, we introduce EAIS, an Execution-Aligned Interaction Sampling strategy that computes sampling signals within an execution-aligned prefix and prioritizes task-progressing and interaction-critical behaviors. We evaluate MATE with representative imitation learning and vision-language-action policies across diverse collaboration tasks. Experiments demonstrate efficient data collection, effective policy learning, and zero-shot transfer from virtual demonstrations to a physical humanoid without real-world fine-tuning. Project page: https://yerik-yu.github.io/MATE/