Visual perception and language guide robots in navigation without maps
Multi-Task Visual Perception Network with LLM Conditioning for Autonomous Navigation
Robotics
Summary
Navigating robots often get lost over time because they rely on maps that can become inaccurate. The authors created a system where robots use local visuals and understand human commands through language to plan their steps instead of relying on a full map. This helps the robot avoid getting lost and safely move around even in places it hasn't been before. Their tests show this method works better than older map-based approaches.
What this means in practice
- •For service robot developers: Build robots that can follow user voice or text commands and reliably navigate indoor spaces without maintaining large global maps.
- •For warehouse automation teams: Deploy navigation systems for robots that adaptively plan local routes using visual cues to reduce navigation errors in complex or changing environments.
Authors
Praveen Kumar, K. R. Guruprasad, Tushar Sandhan
Abstract
Long-term navigation for service robots faces crit- ical challenges like the accumulation of odometry drift and sensor error, which progressively degrade 2D maps and renders traditional path planning algorithms (e.g., A*, RRT*, DiPPer, ViT-A*) ineffective over time. To address this, we propose a user-friendly, interactive framework that eliminates the reliance on globally consistent maps. Our approach integrates visual perception with Large Language Models (LLM) to interpret user commands via text or voice. Instead of relying on a drift- prone global map, the system generates a sequential action plan based on local visual cues and egocentric geometric instructions. These action plans are executed sequentially, allowing the robot to navigate known and unknown environments safely. By reset- ting localization relative to immediate targets, our framework effectively works with a minimum accumulation drift strategy, ensuring accurate, efficient, and collision-free navigation without the maintenance overhead of traditional mapping. Experiments on real-world and simulated data have shown significant improve- ments over other methods. Our source code is publicly accessible at https://github.com/PraveenSingh24/VL-Navigation.