Multi-arm robots use active cameras to improve teamwork success
MAAP: Multi-Agent Active Perception for Collaborative Manipulation
Robotics
Summary
Robots with multiple arms often have cameras on each arm to see what they are doing, but these cameras are usually not used to their full potential. The authors enable each arm’s camera to move and provide new viewpoints while the arms work together, improving how well the team completes tasks. They created a special controller that understands each arm's specific role, helping the robots better decide their actions. Testing in simulations and on physical robot arms showed this approach raised task success rates significantly.
What this means in practice
- •For robotics engineers: Enhance multi-arm robots with dynamic camera viewpoints for higher task success in collaborative manipulation scenarios.
- •For industrial automation designers: Develop collaborative robot systems that improve object placement and handling by integrating active visual feedback from each manipulator arm.
Authors
Bruno N. Y. Chen, Li Kang, Heng Zhou, Xiufeng Song, Zhemeng Zhang, Jiahua Ma, Yiran Qin
Abstract
Multi-agent manipulation naturally produces multiple task-driven viewpoints: every arm carries a wrist camera and moves through the scene while acting. Yet these observations are typically underutilized, and active perception in manipulation is still often treated as requiring a dedicated sensing agent. We introduce MAAP (Multi-Agent Active Perception), in which every arm is dual-purpose: it executes manipulation actions and, through the wrist camera it carries, simultaneously serves as a moving viewpoint for the team. We pair this with RAIL (Role-Aware Imitation Learning), a controller that predicts each arm's current role alongside its action chunk and conditions action generation on it, representing role-dependent actions within one network. Across four simulated tasks, widening the perception regime lifts average success from 56.5% with a fixed camera to 62.5% with one active wrist view and 70.0% with all of them, while MAAP+RAIL reaches 79.2%. RAIL's additional gain is concentrated on the three-arm Microwave task, where success rises from 47% to 82% on identical multi-wrist inputs. On a dual-arm platform, MAAP+RAIL succeeds in 14 of 20 placement trials compared with 0 of 20 for fixed-view ACT. Collaborative manipulation can thus serve as an active perception mechanism in its own right.