Humanoid robots improve parkour skills with better future step prediction
HC2Robot: Hyper-Connection Transformers with Predicted Foresight for Multi-Terrain Humanoid Parkour
Robotics
Summary
Parkour for robots means moving across tricky terrain like stairs and gaps without falling. The authors created a system called HC2Robot that helps a humanoid robot predict where and when it should place its feet next, using only the sensors a robot can carry with it. They use special training techniques that teach the robot to guess these important future steps even though it doesn’t have extra information during actual runs. This helps robots move more safely and successfully over difficult obstacles.
What this means in practice
- •For robotics engineers: Develop perceptive humanoid robots that can navigate complex terrains using predicted foothold timing and location without extra sensors.
- •For search and rescue teams: Deploy humanoid robots that can safely traverse urban rubble and uneven ground by anticipating foothold placement from onboard sensors.
Authors
Wenhao Xu, Yanheng Mai, Yifei Fu, Nan Zheng, Yanbo Zhou, Zirui Huang, Mingsheng Ling, Xinjue Wang, Renjing Xu
Abstract
Humanoid parkour requires robots to reason about future footholds and whole-body motions over highly discontinuous terrain. We revisit the information asymmetry in asymmetric actor-critic learning for perceptive humanoid locomotion: the critic exploits privileged simulator signals during training, whereas the deployable actor operates solely from onboard perception. We argue that the key is not exposing privileged information to the actor, but learning a deployable approximation of it to narrow the information gap between training and deployment. We present HC2Robot, a deployable perceptive humanoid parkour framework consisting of a Predicted Foresight Module (PFM) and a Hyper-Connection (HC) Transformer. PFM predicts from deployable observations the policy-conditioned timing, location, and support quality of each foot's next touchdown event, converting privileged supervision into a deployable latent representation that provides the actor with predicted touchdown information without privileged runtime sensing. The HC Transformer further enhances temporal reasoning through learnable multi-stream routing, while a future motion prediction head provides short-horizon motion priors for regularization. Across stairs, gaps, high platforms, and slopes, HC2Robot achieves a mean simulated traversal success rate of 97.60% and consistently higher safe foothold rates than strong perceptive locomotion baselines, supporting improved traversal performance and geometric foothold support under deployable sensing.