MAMMOTH: A Multi-Modal End-to-End Policy for Off-Road Mobility Robust to Missing Modality
2026-07-14 • Robotics
Robotics
AI summaryⓘ
The authors address the challenge of navigating robots in tough off-road environments where lighting and sensor issues can cause problems. They created MAMMOTH, a system that combines different types of sensor data (like RGB, thermal, 3D points, and speed) and trains the robot to handle missing sensor information. Their method also uses a special policy that helps the robot choose safer and smoother paths. Tests in real off-road settings, including at night, showed that MAMMOTH improves collision avoidance and terrain-aware navigation compared to earlier methods.
Autonomous navigationOff-road roboticsMulti-modal sensor fusionTraversability heuristicDiffusion policyModality dropoutVisual-goal navigation3D pointcloudThermal imagingCollision avoidance
Authors
Ahaan Kotian, Shivani Subramanyan, Suresh Sundaram
Abstract
Reliable autonomous navigation in unstructured off-road environments remains a critical unsolved challenge due to extreme terrain diversity, drastic illumination variations and acute sensor degradation. Recent developments have approached the problem as a traversability costmap estimation or visual navigation task. However, many exhibit heavy reliance on RGB modality, leading to poor performance in varied illumination such as glares, shadows or low ambient light. Achieving robust generalization in such conditions requires integrating modalities that provide supplementary scene information. Such multi-modal methods suffer from a rigid dependency on the presence of near-perfect sensor inputs, leaving them unable to robustly handle sensor degradation or individual modality failure. To address these limitations, we introduce MAMMOTH (MAsking Multi-Modal inputs for Off-road Traversability Heuristic-informed navigation), a unified end-to-end navigation policy for robust off-road visual-goal-conditioned navigation and undirected exploration. Specifically, MAMMOTH efficiently fuses multi-modal observations (RGB, Thermal, 3D Pointcloud and Ego Velocity) and is trained with a modality dropout scheme, enabling it to generalize to missing modalities at inference time. Furthermore, we employ a diffusion policy to learn the joint conditional probability distribution of physically-grounded trajectories and a intrinsic traversability heuristic. MAMMOTH utilizes this heuristic to prefer safer, smoother trajectories. We validate MAMMOTH through extensive real-world robot experiments in distinct off-road environments, including night-time operation. Our results demonstrate superior performance, with significant improvements in collision avoidance, terrain-aware planning and generalization to missing modalities. The code and dataset used for this work will be made publicly available.