GAFT: Geo-Anchored Fine-Tuning for Hazard Identification from Rare Failures
2026-08-31 • Robotics
RoboticsComputer Vision and Pattern Recognition
AI summaryⓘ
The authors address the problem of off-road navigation failure caused by tricky physical spots like getting stuck, which are hard to learn from because failure examples are rare and don't highlight the exact visual reasons. They introduce Geo-Anchored Fine-Tuning (GAFT), a method that improves a vision model by using geometric information to focus attention on important areas while keeping what the model already learned. In tests on a forest hazard dataset, GAFT did better than other methods, showing it can generalize well to new scenarios. This means GAFT helps robots better recognize risky spots without needing lots of detailed failure data.
off-road navigationhigh-centeringentrapmentfine-tuningvision foundation modelgeometry priorLoRA adaptationspatial attention-rolloutleave-one-scenario-outF2 score
Authors
Yanran Xu, Chuanhang Qiu, Yue Wang, Wenbo Wu, Zhaoxing Li
Abstract
Off-road navigation can fail when physical structures induce irrecoverable states such as high-centering or entrapment, requiring human interventions. Identifying these structures is crucial, yet challenging. Such failure events are rare and costly to collect, resulting in limited training data. Moreover, the collected data associate frames with outcomes, but do not indicate the visual cues responsible for the failure. Learning directly from these data can therefore exploit scenario-specific visual cues, leading to poor generalization. We propose \textbf{Geo-Anchored Fine-Tuning (GAFT)}, a parameter-efficient method that adapts a vision foundation model with a geometry-derived prior. It guides LoRA adaptation by aligning a spatial attention-rollout map with the geometry prior, while preserving pretrained representations. On an intervention-verified forest hazard benchmark, across ten independently trained adaptations, GAFT consistently outperforms frozen DINOv2 and supervised PEFT baselines, improving the repeated leave-one-scenario-out mean $F_2$ from 0.0607 to 0.3757 with statistical significance under paired analysis. Within these independently trained models, the best-performing GAFT model achieves a repeated-LOSO $F_2$ of 0.570. Code and benchmark: https://github.com/Xu-Yanran/geo_anchored_fine_tuning