Mobile manipulation improves by aligning perception with subsystems
MoPA: Coordinated Mobile Manipulation via Subsystem-Specific Perception Alignment
RoboticsComputer Vision and Pattern Recognition
Summary
Robots that move around and use arms to manipulate objects need to process different kinds of sensory information for walking and for handling things. The authors developed MoPA, a method that creates separate but coordinated perception parts for movement and arm control, allowing the robot to better understand and act on its environment. This coordination improves the robot’s ability to complete tasks both in simulations and in the real world. MoPA showed higher success rates than previous methods in standard robot benchmarks and real tasks.
What this means in practice
- •For robotics engineers: Develop mobile robots that coordinate base movement and arm manipulation more effectively using separated but coordinated perception streams.
- •For automation integrators: Improve deployment of mobile manipulation systems in real-world settings by enhancing task success rates through subsystem-specific perception-action alignment.
Authors
Guangyu Chen, Qiwei Liang, Shaolong Zhu, Tianxing Chen, Zikuan Xiao, Yifan Xie, Lingfeng Zhang, Ping Luo, Renjing Xu, Wenbo Ding
Abstract
Mobile manipulation requires perceptual evidence at different spatial scales for base motion and arm control, while the two action modalities remain kinematically coupled. Existing policies often employ specialized action generation for different subsystems but condition heterogeneous action branches on a shared perceptual representation, leaving subsystem-specific perception-action correspondence implicit. We present MoPA, a framework that aligns perceptual conditioning with mobility and manipulation while preserving coordination at the action level. Dual Perceptual Streams employ two mutually masked query banks to extract separate perceptual representations from a shared vision-language context. Perception2Action Adaptation jointly updates each query bank and its corresponding action stream at every layer of a structured Mixture-of-Transformers decoder, while enabling information exchange between the two action streams. Coupled conditional flow matching learns a joint vector field for coordinated generation of both action chunks. On the ManiSkill-HAB benchmark, MoPA achieves state-of-the-art performance across all three task suites. Across four real-world tasks, MoPA achieves a mean full-task success rate of 76.3%, outperforming the best baseline by 12.5 percentage points. Ablation studies and further analyses validate the effectiveness of the proposed design. Website is available at: https://mopa-policy.github.io/.