Latent motor adaptation improves coordinated robot hand actions
Scaling Latent Motor Adaptation to Coordinated Bimanual Manipulation
Robotics
Summary
Robots have trouble handling objects when things like weight or shape change. The authors studied a technique called latent motor adaptation that helps robots adjust their actions based on what they’ve experienced so far. They tested this method on tasks where two robot arms work together to pick up and move objects with different properties. Their approach worked well for short tasks but still struggled with longer sequences like stacking. The results also showed this method can be used directly on real-world robots without extra training.
What this means in practice
- •For industrial robot engineers: Control two-armed robots to handle objects with varying properties more reliably in manufacturing tasks.
- •For robotics system integrators: Deploy bimanual robotic manipulation policies learned in simulation directly onto real hardware without real-world tweaks.
Authors
Seung-Bin Joo, Alessandro Albini, Luca Wiehe, Perla Maiolino, Daniel Olmeda Reino, João F. Henriques
Abstract
Robust manipulation under uncertain object dynamics and geometry remains a core challenge in real-world robotics. Policies trained under fixed assumptions often degrade when object mass, friction, scale, or shape vary. Embedding-based adaptation methods such as Rapid Motor Adaptation (RMA) address this by conditioning actions on latent representations of environment dynamics inferred from interaction history. We investigate whether this paradigm scales from single arm settings to coordinated bimanual manipulation, where higher dimensional control, coupled contacts, and shared object forces introduce additional complexity. We evaluate bimanual RMA on three contact-rich ManiSkill tasks with randomized YCB object properties. Bimanual RMA, using observation-action history and dual wrist depth images, retains privileged policy performance on bimanual picking and lifting, while an otherwise matched stationary PPO policy without latent conditioning achieves low success; all methods remain limited on long-horizon stacking. We further demonstrate direct sim-to-real transfer on a dual Franka lifting task without real-world finetuning. Together, these results suggest that latent conditioning provides useful structure for scaling robust control to coordinated bimanual systems, while longer horizon tasks require additional temporal structure.