Humanoid robot arms learn to work together using new teleoperation system
Compositional Humanoid Manipulation through Isomorphic Teleoperation and Independent Skill Routing
Robotics
Summary
Humanoid robots need to use their arms in many different ways, but teaching every possible movement is too hard. The authors created a new method that lets robots combine arm skills learned separately by controlling each arm independently but sharing some knowledge. They also built a special exoskeleton that helps humans directly control robot arms more reliably than virtual reality controllers. This new setup improved the robot’s ability to open a cabinet by using better demonstrations and smarter skill switching. Their system shows more successful teamwork between robot arms across different positions than previous methods.
What this means in practice
- •For robotics engineers: Enable humanoid robots to perform complex arm tasks by combining separately trained skills and improving teleoperation input quality.
- •For industrial automation teams: Use enhanced humanoid manipulation to automate complex cabinet or workspace operations requiring whole-body contact and repositioning.
Authors
Yiming Wang, Peixian Yang, Lifeng Zhuo, Lai Wei, Xinling Du, Kun Song, Weiming Wang, Daolin Ma
Abstract
Humanoid manipulation requires reusable arm skills, yet demonstrating every possible skill combination is impractical. We present Humanoid Semantic Mixture-of-Experts Diffusion Policy (HS-MoDP), which composes separately demonstrated arm skills through independent semantic routing over a shared expert pool. This design preserves arm-specific skill control while sharing action-generation parameters across arms. A semantic scheduler integrates the learned policy with an existing locomotion controller through alternating manipulation and repositioning stages, extending the effective workspace and enabling skill reuse across location changes. To acquire demonstrations of whole-body contact manipulation, we develop G1Exo, an isomorphic exoskeleton that provides direct joint-level control of arm configurations. Experiments on a Unitree G1 show stronger skill composition than the baselines. G1Exo enables more reliable contact teleoperation than end-effector VR-IK, and its demonstrations improve contact-skill learning across four policy architectures. For HS-MoDP, switching from VR-IK to G1Exo demonstrations raises full cabinet workflow success from 35% to 65%. All project hardware designs and source code will be released as open source after the review process.