World model improves off-road robot navigation using terrain data and physics
Verti-WM: A Physics-Aided Exteroceptive World Model for Off-Road Reinforcement Learning
Robotics
Summary
Off-road robot navigation needs lots of information about how a vehicle interacts with rough terrain, which is expensive to collect in detailed simulations. The authors created Verti-WM, a model that combines physics rules with terrain data from maps to better predict how a robot will move on different surfaces. This improves prediction accuracy and lets robots learn navigation policies much faster than using full simulations. They also showed that policies trained with Verti-WM work well on a real 4-wheeler robot, with higher success rates than just transferring from simulation.
What this means in practice
- •For robotics engineers: Train off-road robot control policies efficiently using a combined physics and terrain model instead of high-cost simulators.
- •For autonomous vehicle teams: Improve real-world autonomous navigation performance by optimizing vehicle policies within a world model conditioned on terrain data.
Authors
Chenhui Pan, Tong Xu, Xuesu Xiao
Abstract
Reinforcement learning for off-road navigation requires extensive vehicle-terrain interaction data, which are costly to collect in high-fidelity simulation. World models offer a promising alternative by replacing simulator roll-outs during policy optimization. However, an off-road world model must condition state transitions on exteroceptive terrain information, which proprioception alone does not provide. This challenge is further amplified by the need to model both rigid and deformable terrain, where data-driven and physics-based approaches offer complementary strengths. We propose Verti-WM, a physics-aided exteroceptive world model that recurrently fuses a frozen Transformer for rigid terrain and a neuro-symbolic terramechanics model for deformable terrain. Elevation and semantic observations queried from a supplied map at each predicted pose condition fusion, enabling six-degree-of-freedom rollouts for policy optimization without further simulator access. Verti-WM reduces prediction error by 34.6% and 21.7% over data-driven and physics-based baselines, respectively. Policies trained entirely within Verti-WM achieve comparable task success rates while reducing computation time by 23.6X relative to direct training in the high-fidelity simulator. We further validate Verti-WM using real-world data, enabling policy optimization within learned real-world kinodynamics and achieving a 80% success rate on the Verti-4-Wheeler platform, compared with 40% for direct sim-to-real transfer.