Language guided robots adapt instantly with one phone scan
One Scan is Enough: Demonstration-Free Adaptation for Language-Guided Navigation
RoboticsArtificial IntelligenceComputation and LanguageComputer Vision and Pattern RecognitionMachine Learning
Summary
Robots that navigate indoors often struggle when placed in new buildings because the layouts and appearances differ from what they learned before. The authors introduce a method that allows these robots to quickly adapt by scanning the new environment once using a phone, without needing detailed human demonstrations. This single scan creates a virtual map and instructions automatically, letting the robot retrain itself more effectively. The new approach significantly improves success rates in real buildings compared to older methods relying on human input.
What this means in practice
- •For robotics deployment teams: Adapt indoor navigation robots to new buildings instantly without collecting human demo data.$Commercial implications: Enables companies to sell ready-to-deploy indoor navigation robots that require only a quick phone scan to work in new environments.
- •For facility management teams: Use one quick scan to configure robot guides or assistants for new or remodeled buildings without extra training.
Authors
Zhuoyuan Yu, Yuxing Long, Du Wang, Junyang Wang, Hongwei Fan, Hao Dong
Abstract
Deploying a language-guided navigation policy in a new building usually means watching it fail. Public-benchmark VLAs degrade scene by scene as layout, appearance, and language drift, and the standard fix of teleoperating fresh trajectories on site does not scale. We replace human demonstration with a single brief iPhone scan. Scan2Instruction compiles that scan into a complete training environment, automatically synthesizing executable navigation episodes with grounded instructions. PRPT-KS then adapts the policy through a closed-form Pareto utility read directly off the reconstructed scene, with no learned critic and no human labels. Across five real environments, the two designs together drive our RealNav to an 85% real-robot success rate, compared with 49% for a RealNav baseline post-trained on human demonstrations, and still achieve 82% in unscanned regions of the same buildings, compared with 41% for the same baseline. Our work turns demonstration-free scene adaptation for language-guided navigation from an aspiration into a working reality.