Vision and language navigation improves by revisiting memory use
ReInsVLN: Revisiting Historical Evidence for Training-Free Vision-and-Language Navigation
Robotics
Summary
In vision-and-language navigation, a system must remember and use past observations to follow instructions successfully. The authors show that reviewing earlier information in a smart way can help fix mistakes made during navigation without needing extra training. They create a new method called ReInsVLN that compresses visual context to keep important history and reminds the system to revisit these memories on a schedule. This method boosts navigation success rates and reduces the computer resources needed, even working well on real robots.
What this means in practice
- •For robotics engineers: Improve robot navigation systems by integrating memory revisiting techniques to increase navigation accuracy without extra model training.
- •For ar/vr interface developers: Enhance sequential instruction following in AR/VR environments by using compact visual history that guides users through complex tasks more effectively.
Authors
Siwei Cheng, Hao Wu, Zitong Qi, Xinming Zhang, Haoyu Xi, Peiran Yin, Haozhe Hu, Xudong Wang, Xiaoyu Shen, Wei Zhang
Abstract
Vision-and-language navigation (VLN) relies on retaining and using relevant observations. Yet growing histories introduce redundancy, and retained information may remain underused. We introduce ReInsBench, a diagnostic benchmark that examines history utilization through controlled continuation trials. Its results show that history-review reinstruction can recover navigation failures using the existing task context. Motivated by this finding, we propose ReInsVLN, a training-free framework supporting memory construction and utilization in frozen navigators. Coverage- and Instruction-Aware Compression constructs compact visual context through temporal redundancy reduction, coverage-aware protection, and instruction-guided token selection. Forgetting-Inspired History Reminders guide navigators to revisit retained observations according to a predefined schedule, without additional model calls. On R2R Val-Unseen, ReInsVLN improves success rates by 8.48 and 3.37 percentage points over the native StreamVLN and InternVLA-N1, reaching 64.27% and 64.44%, respectively. Visual compression largely preserves navigation performance while reducing prefill latency and KV cache memory by up to 21.77% and 32.25%, respectively, on matched observation workloads. Real-world trials on humanoid and quadruped robots further support the applicability of ReInsVLN to sequential instruction following and ordinal navigation.