Mars rover learns to recognize safe terrain for driving using images and sensors
Learning to Drive on Mars: Visual Multimodal Traversability Estimation for Off-World Navigation
Robotics
Summary
Driving safely on Mars is hard because the terrain looks very different and can be tricky to navigate. The authors created a large dataset from 45 kilometers of Mars rover drives, using cameras and sensors to capture details about the ground and rover's movements. They then developed a new computer method that uses this data to better predict which areas are safe for the rover to drive on. Their method works better than previous ones and can be used to help real rovers plan safer routes on Mars.
What this means in practice
- •For space mission engineers: Develop more reliable rover navigation systems for Mars by integrating improved terrain safety predictions from multimodal sensor data.
- •For autonomous vehicle developers: Use Mars rover visual and sensor data techniques to enhance terrain assessment for autonomous off-road vehicles on Earth.
Authors
Darren Chiu, Cole Wilson, Andrei Tumbar, Gaurav S. Sukhatme, Steven Myint
Abstract
Autonomous navigation on Mars requires vehicles to distinguish between traversable terrains across diverse and visually challenging environments. However, progress in learning-based navigation for off-world environments has been limited by the lack of large-scale datasets. Since landing in Jezero Crater, the Mars 2020 Perseverance rover has traversed terrain ranging from sandy dunes, rocky patches, and flat bedrocks. As a result, this paper presents a dataset spanning 500 sols and 45km of trajectories driven by both human operators and the onboard planner, ENav. Our dataset contains grayscale stereo image pairs, poses, accelerometer readings, rocker-bogie angles, and estimates of tilt and wheel slip. Building on this dataset, we introduce an uncertainty-aware traversability-estimation framework that learns terrain representations from multimodal driving experience. We compare our proposed method against existing approaches on the Mars 2020 dataset and show that our method achieves an AUROC of 0.874 and an F1 score of 0.758, outperforming the strongest baseline by 0.058 and 0.156, respectively, while also achieving the highest average precision and recall. Finally, we show that the visual representations can be integrated into path planners, such as ENav, on a physical rover test bed. Videos, code, and the M2020 dataset will be available at https://darren-chiu.github.io/learning-to-drive-on-mars.