Summary
Putting tiny parts together is hard because robots need very precise movements and must understand forces while touching objects. The researchers found that measuring a robot’s own joint positions and movements is more reliable than relying on cameras or force sensors, which can look different in simulation versus real life. They created a method that teaches a robot to connect what it sees and feels to its own joint movements, ignoring distracting differences like lighting or sensor quirks. This allows robots to practice in simulation and then succeed in real-world assembly tasks without extra adjustments. Their method worked well in several tests and handled disruptions better than other approaches.
Sim-to-real transferProprioceptionForce/torque sensingReinforcement learningRobot assemblyDomain adaptationCross-modal learningSimulationPose estimationRobotic manipulation
Authors
Yuhan Wang, Yurou Chen, Hongye Jiang, Wenzhao Lian
Abstract
Contact-rich assembly remains challenging because it requires submillimeter spatial accuracy and reliable interpretation of forces during sustained contact. Although simulation-based reinforcement learning offers a scalable training paradigm, discrepancies in visual observations, contact dynamics, and force/torque (F/T) measurements often limit policy transfer. We observe that proprioception is comparatively consistent across domains because calibrated joint positions and consistently computed joint velocities align closely between simulation and hardware. Based on this observation, we present PACE (Proprioception-Anchored Cross-Modal Encoder), which supervises temporal visual and F/T representations by predicting proprioceptive state transitions. Static domain-specific factors, including lighting, texture, and sensor bias, contain little information about joint motion; the proposed objective therefore encourages the encoder to suppress these factors while retaining task-relevant motion cues. Policies trained on frozen PACE features are deployed on hardware without real-world fine-tuning or object-pose tracking. Across four contact-rich assembly tasks, PACE attains an average real-world success rate of 93.3\% and only a 2.7-percentage-point sim-to-real drop, meanwhile remaining robust to perturbations that substantially degrade pose-based and learned-fusion baselines.