New reward method improves robot walking by matching motion paths
Flow-Matched Motion Priors: Online Optimal-Transport Rewards for Imitation Learning
Robotics
Summary
Teaching robots to move like humans or animals requires a way to guide them toward good motions seen in examples. The authors improve on previous methods by creating a reward that considers whole paths of motion rather than just single poses or averages. This reward is learned online and helps the robot walk more stably and naturally. Tests show fewer falls and better walking speeds compared to older approaches.
What this means in practice
- •For robotics engineers: Develop controllers that help robots imitate complex walking behaviors more reliably and naturally from demonstration data.
- •For animation software developers: Generate realistic human or animal motion sequences by guiding animations using demonstrated motion paths rather than static poses.
Authors
Yilin Zou, Chenghua Liu, Chenglong Wu, Fanghua Jiang
Abstract
Learning a motion prior requires a reward that guides a policy from its current behavior toward demonstrated motion. Adversarial Motion Priors (AMP) provide such a reward with a discriminator. However, adversarial objectives can become uninformative when policy and expert supports are far apart. A naive use of optimal transport (OT) averages matched expert successors into a barycentric target. Averaging across gait phases can weaken the target's joint motion. We introduce Flow-Matched Motion Priors (FMP), an online scalar reward learned from paths connecting current rollout histories to an expert motion bank. Entropic OT supplies the coupling. Before each policy update, we train a neural potential with flow matching (FM) along the rollout-to-expert paths, endpoint-gradient supervision, and relative-value calibration. The actor receives only physical observations and the reward remains a scalar, as in AMP. Controlled reward-model experiments show substantially better generalization beyond the fitting rollout than value-only or endpoint-only fitting. On Unitree G1, matched 50-million-transition experiments compare FMP with AMP, a barycentric OT reward, and nested ablations under demonstration and fixed-pose initialization. FMP produces stable forward walking at 0.727 m/s from demonstration resets and 0.338 m/s from a fixed default pose. In the fixed-pose condition, it incurs 129 falls versus 243 for the endpoint-only control. Against a static score-gradient teacher, dynamic FM reduces score-increment error at interpolation fractions 0.25 and 0.50 while using 29% less offline fitting time.