Humanoid robots improve obstacle crossing with dual models
DWMP: Leveraging Dual World Models for Humanoid Obstacle Traversal
Robotics
Summary
Humanoid robots need to move through areas full of obstacles using information from their own sensors and cameras. The authors found that treating the robot’s movement data and visual data separately helps because they have very different qualities. They built a system that uses one model to simplify the robot’s movement info and another to compress the camera images. Combining these helps the robot decide its next move more effectively. Tests in simulation and on a real robot showed better obstacle crossing with this approach.
What this means in practice
- •For robotics engineers: Improve humanoid robot navigation in cluttered environments by using separate models for motion dynamics and visual perception.
- •For autonomous vehicle developers: Enhance obstacle detection and response by separately modeling vehicle dynamics and sensor data for better decision making.
Authors
Rongjun Jin, Jianming Ma, Yue Gao
Abstract
Humanoid robots must traverse cluttered obstacle fields using onboard proprioceptive and visual observations, yet existing methods usually process multimodal observations without explicitly considering their different characteristics: proprioceptive observations are low-dimensional but governed by highly nonlinear robot dynamics, while egocentric visual observations are high-dimensional, noisy, and redundant. We propose DWMP (Dual World Model Policy), a framework that provides the actor with separate but complementary world-model representations for humanoid obstacle traversal. A Koopman-based dynamics world model lifts proprioceptive observations into a latent space where their temporal evolution is approximately linear, making the dynamics features easier for the actor to learn from. An RSSM-based visual world model compresses egocentric depth observations into compact stochastic states while preserving obstacle-related geometry. The student policy receives the fused latent representation for action generation, combining linearized proprioceptive dynamics with compressed visual perception. Experiments in simulation and on a Unitree G1 humanoid robot show that DWMP improves obstacle traversal performance over baselines and supports real-world deployment under randomized obstacle layouts.