Gated residual coordination improves humanoid robot hand and body movement
Gated Residual Body-Hand Coordination for Whole-Body Humanoid Teleoperation
Robotics
Summary
Controlling a humanoid robot's whole body and hands at the same time is tricky because the commands for the body and hands often don't line up perfectly. The authors developed a system that applies careful corrections to keep the body and hands working together naturally without changing the main control parts. This helps reduce errors in the robot’s wrist and finger positions, making its movements more coordinated while still following the original motion. They tested their method in simulations and showed it can improve the robot's interaction with objects using just motion and internal sensing, without needing extra information about what it’s touching.
What this means in practice
- •For robotics engineers: Coordinate body and hand motions of humanoid robots in teleoperation tasks to improve interaction accuracy without redesigning core controllers.
- •For robot software developers: Enhance existing humanoid robot control software by integrating residual correction policies that fix geometric mismatches in bimanual coordination.
Authors
Ruiming Wu, Shuang Li, Liding Zhang, Alois Knoll, Zhaopeng Chen
Abstract
Whole-body humanoid teleoperation commonly combines a motion-tracking policy with a separate dexterous-hand retargeter. However, independently generated commands do not explicitly preserve body-hand geometric relations, leading to mismatches in relative wrist poses and fingertip positions during bimanual interaction. We present a gated residual coordination framework that keeps both modules frozen and applies bounded corrections to their outputs. A motion-conditioned action gate allocates correction authority across joint groups, while reference-geometry-dependent reward gates emphasize relevant interaction objectives during training. To establish the nominal body controller on Agile One, we introduce multi-pose morphology calibration that jointly estimates triaxial scales and effector-local offsets, together with staged motion dataset curation for training a SONIC-based tracker. The residual policy uses human motion references, initial commands, and robot proprioception without explicit object or contact observations. In simulation, it reduces wrist and fingertip geometry errors by 39.2-56.3% over direct composition on held-out GRAB motions, while preserving whole-body tracking on AMASS, with success rates of 89.03% without residual coordination and 89.29% with it. Ablations characterize the contributions of reward gating, adaptive correction authority, and separate body and hand correction heads.