Proprioception improves robot assembly from simulation to reality
Zero-Shot Sim-to-Real Contact-Rich Assembly via Proprioception-Anchored Cross-Modal Pretraining
RoboticsArtificial Intelligence
Summary
Contact-rich assembly tasks, like snapping parts together, are hard for robots because they need super precise movements and must understand forces while touching objects. The authors found that a robot's sense of its own joint positions and movements, called proprioception, stays consistent between simulations and the real world. They created a system named PACE that uses this reliable proprioceptive data to better learn how visual and force information relates to movement. Robots trained with PACE worked well in real assembly tasks without extra tuning, achieving over 90% success and staying strong even when things changed unexpectedly.
What this means in practice
- •For robotics engineers: Train assembly robots in simulation and deploy them directly on hardware with high success despite real-world differences.
- •For industrial automation teams: Improve factory robot reliability for tasks needing precise contact by using proprioception-anchored training without extra real-world data.
Authors
Yuhan Wang, Yurou Chen, Hongye Jiang, Wenzhao Lian
Abstract
Contact-rich assembly remains challenging because it requires submillimeter spatial accuracy and reliable interpretation of forces during sustained contact. Although simulation-based reinforcement learning offers a scalable training paradigm, discrepancies in visual observations, contact dynamics, and force/torque (F/T) measurements often limit policy transfer. We observe that proprioception is comparatively consistent across domains because calibrated joint positions and consistently computed joint velocities align closely between simulation and hardware. Based on this observation, we present PACE (Proprioception-Anchored Cross-Modal Encoder), which supervises temporal visual and F/T representations by predicting proprioceptive state transitions. Static domain-specific factors, including lighting, texture, and sensor bias, contain little information about joint motion; the proposed objective therefore encourages the encoder to suppress these factors while retaining task-relevant motion cues. Policies trained on frozen PACE features are deployed on hardware without real-world fine-tuning or object-pose tracking. Across four contact-rich assembly tasks, PACE attains an average real-world success rate of 93.3\% and only a 2.7-percentage-point sim-to-real drop, meanwhile remaining robust to perturbations that substantially degrade pose-based and learned-fusion baselines.