Humanoid robots learn to walk and move better on tricky surfaces
PGMT: Perceptive General Motion Tracking for Humanoid Robots
Robotics
Summary
Humanoid robots often struggle to walk and move well on uneven or complicated ground because their usual motion guides don't work there. The authors created a method called PGMT that helps these robots adapt their movements by using what they see around them to adjust how they move. Their system focuses on important parts of the terrain related to the robot's current actions and allows small changes while keeping the original motion goal. They tested PGMT on a real robot that could walk over obstacles and recover from falls without extra training. This helps robots move in more natural and flexible ways on different surfaces.
humanoid robotsmotion trackingterrain adaptationteleoperationlocomotionmotion referencefall recoveryreal-world terrain
Authors
Hongyi Li, Li Peizhuo, Yucheng Tao, Ze Wang, Fangzhou Xu, Jinyi Chen, Yanyan Yuan, Dapeng Jia, Yongbin Jin, Mingfeng Fan, Guillaume Sartoretti, Hongtao Wang
Abstract
Humanoid motion trackers can reproduce diverse whole-body motions, but their performance degrades on complex terrain where terrain-agnostic references become physically infeasible. We present PGMT, a Perceptive General Motion Tracking pipeline for humanoid robots that learns terrain adaptation from independently selected motion references and terrains. PGMT first learns a general tracking and recovery prior, then incorporates terrain perception through motion-conditioned terrain glimpses that selectively encode regions relevant to the current motion. Terrain-aware tracking relaxation allows necessary deviations from the reference while preserving its motion intent. Zero-shot deployment on a Unitree G1 demonstrates robust terrain-adaptive locomotion and whole-body motion execution over real-world terrain with obstacles up to 37 cm high, while supporting teleoperation, dynamic motion tracking, and fall recovery. PGMT extends general humanoid motion tracking beyond flat ground, providing a unified policy for terrain-adaptive locomotion, diverse whole-body behaviors, and teleoperation in complex environments.