NavPatch improves robot obstacle maps using vision and language models
NavPatch: Evidence-Guided Object-Level Costmap Correction with Vision-Language Models
Robotics
Summary
Mobile robots use maps to avoid obstacles, but these maps often don't perfectly show how different objects affect movement. For example, some cables or curtains might be missed or wrongly marked as obstacles. The authors created NavPatch, a system that corrects these maps by recognizing and adjusting obstacle shapes and sizes based on what a robot sees and understands through combined vision and language analysis. This approach led to more successful robot navigation in real-world tests.
What this means in practice
- •For robotics engineers: Enhance robot path planning by updating obstacle maps with object-level corrections informed by combined visual and language processing.
- •For warehouse automation teams: Improve mobile robot navigation reliability by correcting obstacle maps dynamically in diverse environments with vision-language supported updates.
Authors
Shiji Sun, Xingyu Tao, Hao Wang, Ling Wang, Zhengyi Chen
Abstract
Mobile robots typically rely on geometric maps for obstacle avoidance and path planning, but the resulting obstacle representation does not always match how an object should affect navigation. A low lying cable may be missed, a flexible curtain may create spurious blockage, and a traffic cone may require an exclusion region larger than its observed footprint. We present NavPatch, an object level correction layer that assigns ADD, REMOVE, or EXTEND to navigation relevant object categories through periodic scene understanding with a vision-language model. Open vocabulary grounding localizes object instances, and LiDAR and RGB-D observations provide 3D support. Observation quality filtering and cross frame maintenance determine when each correction patch is committed, replaced, or revoked. In 50 real robot trials across five layouts, NavPatch achieves an overall success rate of 86.0%. An ablation study of four configurations with 200 runs in total shows that NavPatch improves the success rate from 70.0% to 86.0% and reduces the false commit rate from 68.4% to 40.7% compared with updates based only on the current observation.