LiDAR based learning system improves legged robot terrain navigation
JEPLO: Joint-Embedding Predictive Learning for LiDAR-Based Legged Locomotion
Robotics
Summary
Navigating rough ground is tricky for legged robots because they need to understand their surroundings well. The authors developed a new learning system called JEPLO that uses LiDAR sensors to sense the environment without needing to build maps first. This system helps robots predict and adapt to different terrains like stairs and obstacles, even when the sensor data is noisy or incomplete. Their method works well when moving from simulations to real robots, allowing smoother and more reliable walking in challenging places.
What this means in practice
- •For robotics engineers: Develop legged robots that navigate complex terrains without creating maps by using raw LiDAR data and onboard predictive learning.
- •For autonomous vehicle developers: Enhance LiDAR-based perception systems to better handle occlusions and noisy data when maneuvering over uneven surfaces.
Authors
Qihao Yuan, Yixuan Qiu, Ziyu Cao, Ming Cao, Kailai Li
Abstract
Light detection and ranging (LiDAR) remains less explored than RGB-D sensing for perceptive legged locomotion, and existing LiDAR-based approaches often rely on explicit mapping. We present JEPLO (Joint-Embedding Predictive learning for legged LOcomotion), a single-stage learning framework for mapping-free, LiDAR-based perceptive locomotion for legged robots. We introduce a proprio-exteroceptive JEPA (PE-JEPA) world model to learn predictive egocentric terrain representations from onboard observations, including raw LiDAR scans. A concurrent JEPA-teacher-student (CJTS) pipeline is further proposed to train a locomotion policy informed by JEPA latent representations in simulation using deep reinforcement learning with a simple reward formulation. The framework achieves successful sim-to-real transfer, enabling omnidirectional traversal of diverse terrains, including long staircases and high boxes, with lightweight onboard computation. Evaluations demonstrate greater robustness than existing perceptive locomotion frameworks, particularly under degraded perception caused by occlusion, sparsity and noise. Further analysis validates JEPLO's ability to retain task-relevant information under these challenging conditions. We open-source our implementation, experimental datasets, and hardware setup designs https://github.com/ASIG-X/JEPLO.