Large language models enable better robot navigation using standard map images
Spatial and Semantic Reasoning for LLM-Driven Robot Navigation via MCP
Robotics
Summary
Robots use maps to move around, but the raw map data is hard for language models to understand directly. The authors created a new way to convert robot maps into easy-to-read images and notes that language models can use to make navigation decisions. Their method works with usual robot software and doesn't need special robot-specific programming. In tests with a robot simulator, language models guided the robot to cover maps fully and followed natural language instructions accurately.
What this means in practice
- •For robot software developers: Integrate language models for spatial and semantic navigation decisions without modifying core robot navigation systems.
- •For simulation engineers: Use standardized map image representations to test language model-driven navigation in virtual indoor environments.
Authors
Jungsoo Lee, Jaegyun Park, Wansoo Kim
Abstract
Large language models (LLMs) are increasingly used as natural-language interfaces for robotic systems, yet their integration with Robot Operating System (ROS)-based navigation remains limited by two gaps. First, navigation data such as occupancy grids are represented as raw geometric messages that are difficult for LLMs to use directly as spatial or semantic context. Second, adding LLM-driven capabilities often requires custom wrappers or robot-specific interfaces, limiting reuse across systems. To address these challenges, we propose a non-invasive framework that connects LLM reasoning with ROS-based navigation through a navigation-oriented representation layer, exposed through the Model Context Protocol (MCP) as standardized, reusable tools so that any MCP-compatible LLM can access them without robot-specific wrappers. The visual map modules transform occupancy grids into metric, pose-aware images for goal reasoning, while the semantic annotation modules record waypoint-level observations with robot poses. We evaluate the framework on three tasks: autonomous mapping, spatial reasoning-based navigation, and semantic reasoning-based navigation. The results show that the evaluated LLM backends use these representations to achieve over 97% map coverage and select spatial or semantic navigation targets from natural-language instructions in a simulated indoor environment. This demonstrates representation-mediated LLM navigation without modifying the existing ROS navigation stack.