Hyperspectral imaging tested with 3D SLAM for planetary rover navigation
HyperLoop: Hyperspectral-Assisted Loop Closure Detection in 3D-SLAM for Planetary Exploration
Computer Vision and Pattern RecognitionRobotics
Summary
Navigating robots on planets like the Moon or Mars is hard because many places look very similar and featureless. This paper explores combining hyperspectral imaging, which looks at detailed material properties of surfaces, with a 3D mapping method to help robots recognize places they’ve visited before more reliably. The study found that with the kind of light sensors used (600-860 nm), hyperspectral data didn't improve location recognition much beyond existing methods using color images. The authors suggest future work should use sensors capturing different types of light to better tell planetary materials apart.
What this means in practice
- •For planetary rover engineers: Improve loop closure detection methods by integrating hyperspectral data for navigating featureless extraterrestrial terrain.
- •For remote sensing developers: Design spectral sensors and descriptors for robotic navigation focused on minerals beyond the Red-NIR range, informed by this study’s findings.
Authors
Annie Bhalla, Dennis Rotondi, Riccardo Giubilato, Kai O. Arras, Wolfgang Stuerzl
Abstract
Autonomous navigation of planetary rovers is challenged by visually repetitive, featureless terrain. SLAM systems based on 3D Gaussian Splatting (3DGS) offer dense mapping but rely on RGB appearance descriptors that degrade under perceptual aliasing - a common failure mode in extraterrestrial environments where physically distinct locations appear visually identical. This paper presents HyperLoop, the first framework integrating hyperspectral imaging (HSI) into the loop closure module of a 3DGS-based SLAM system. Hyperspectral descriptors encode material-level spectral signatures complementing RGB features, providing robustness to illumination changes and visual repetitiveness. Two HSI descriptor families are introduced: Bag-of-Spectral-Words (BoSW), encoding spectral material distributions via TF-IDF weighted histograms, and PCA-K VLAD (PCAK-VLAD), capturing first-order residual statistics of PCA-compressed spectral signatures. Both are evaluated with SIFT-VLAD and NetVLAD RGB descriptors under Strict-AND and Weighted fusion strategies on a custom dataset from the Moon-Mars Outdoor Test Site (MMOTS) at DLR, comprising synchronized RGB-D and hyperspectral sequences across four planetary analog traverses (65.5 m total). A key finding is that the sensor's Red-NIR range (600-860 nm) provides limited discriminability for rock terrain: the best HSI fusion (SIFT-VLAD + BoSW, Strict-AND) achieves 5.04 cm ATE RMSE, within 1.8% of SIFT-VLAD alone (4.95 cm). The gain over NetVLAD (8.81 cm) is attributable to SIFT-VLAD, not HSI fusion. This motivates future adoption of SWIR sensors capturing diagnostic mineral absorption features absent from the 600-860 nm range for planetary SLAM missions.