Vision navigation improves by guiding attention and spatial cues
GuidedNav: Shaping Vision-Language Navigation Representations through Pre-action Attention and Spatial Guidance
Computer Vision and Pattern Recognition
Summary
Vision-and-language navigation is about helping a robot or computer move by following spoken or written directions. The authors present GuidedNav, a method that helps the system focus on the right landmarks and understand the space around them better while it decides what to do next. This focused guidance improves how well the system can follow instructions and find its way, especially in new places not seen before. The authors also created new datasets with added annotations to help make future studies more understandable.
What this means in practice
- •For robotic navigation developers: Develop navigation systems that better use visual landmarks and spatial context to follow instructions more reliably in unfamiliar environments.
- •For augmented reality app engineers: Create AR navigation aids that improve spatial understanding and landmark recognition from standard camera input for indoor or outdoor wayfinding.
Authors
Jingyu Guo, Jiaxin Huang, Chengxing Lin, Xiaoyu Luo, Cheng Wen, Shanshan Ye, Tingjin Chu, Shaoli Huang, Mingming Gong
Abstract
Vision-and-language navigation (VLN) requires an agent to track progress through an extended instruction, identify landmarks relevant to its current objective, and understand the surrounding spatial structure to act. End-to-end navigation policies typically rely on action supervision to implicitly learn these capabilities. Without explicit guidance on the evidence underlying each decision, however, policies can overfit to correlations predictive of training actions rather than learn the task-relevant grounding and geometry needed for generalization. Our insight is to guide the representations that connect instructions to actions: identifying what remains to be done determines which visual evidence matters, while understanding the geometry around that evidence informs how to move. We introduce GuidedNav, a unified framework that explicitly shapes these representations through training-time guidance. Subtask and landmark guidance steer native pre-action attention toward the active instruction segment and relevant visual regions, grounding action prediction in the current objective. Spatial guidance aligns the policy's internal representations with geometric features that capture long-horizon context and local spatial structure. Together, these objectives strengthen navigation capabilities while avoiding the inference overhead of intermediate textual reasoning and online geometric encoding. We also introduce Guided-R2R and Guided-RxR, two annotation-enriched navigation datasets that support more interpretable VLN research. GuidedNav establishes state-of-the-art performance among the compared monocular RGB policies, improving success rates by 8.1% and 13.1% relative to the strongest prior results on R2R-CE and RxR-CE val-unseen, respectively, while achieving faster inference. Extensive experiments further demonstrate the effectiveness and real-world transferability of our approach.