Learning Reusable Hybrid Motion Priors for Humanoid Locomotion from Motion Imitation
2026-07-27 • Machine Learning
Machine LearningRobotics
AI summaryⓘ
The authors present a three-step method to teach robots to walk and move more like humans by first training a policy to copy human motion, then compressing that knowledge into a reusable codebook, and finally training task-specific controllers that pick actions from this codebook. This approach allows robots to learn different walking tasks without retraining from scratch each time. They tested their method in simulations and on a real robot, finding it keeps the original motion quality and can adapt to various tasks. They also improved the system’s organization and stability by using a special training technique called the rotation trick.
reinforcement learninghumanoid locomotionmotion imitationpolicy distillationvector quantizationcodebookproprioceptive encodervelocity trackingpoint-goal navigationrotation trick
Authors
Valerio Belli, Valerio Modugno, Enrico Mingo Hoffman, Fabio Amadio
Abstract
Reinforcement learning can produce robust humanoid controllers, but each new task is typically trained as a separate policy with its own reward design and training process. Motion imitation provides an alternative source of motor competence by training policies to track retargeted human motions, yet the resulting controllers remain reference trackers and are not directly usable as task policies. We propose a three-stage pipeline that turns motion-imitation skills into a reusable hybrid motion prior (HMP) for humanoid locomotion. First, an expert policy is trained to imitate retargeted human motion-capture clips. Second, the expert is distilled into a frozen architecture composed of a proprioceptive encoder, a residual vector-quantized (RVQ) codebook, and an action decoder. Third, task-level policies are trained to solve locomotion tasks by selecting discrete codebook entries while the HMP remains frozen. We evaluate the method on velocity tracking, point-goal navigation, and fall-recovery velocity tracking in simulation, and deploy the velocity-tracking policy on a real Unitree G1 robot. The distillation process preserves the tracking behavior of the expert, while the resulting HMP can be reused without retraining as the action interface for different downstream locomotion policies. The learned HMP reveals an interpretable codebook structure in which the number of active RVQ stages modulates the available gait patterns. We further show that training the codebook with the rotation trick improves latent organization and reduces downstream falls compared with a standard straight-through estimator.