OptiSight: Bridging Semantic Reasoning and Geometric Control for Embodied Navigation
2026-08-24 • Robotics
RoboticsComputer Vision and Pattern Recognition
AI summaryⓘ
The authors introduce OptiSight, a new method for robots to navigate indoors by combining language understanding and precise movement control. Their system uses a vision-language model (VLM) to recognize objects when needed, and a geometric control method to guide the robot continuously without heavy mapping. This approach works well in different indoor settings, avoiding obstacles and handling unclear instructions, all while using limited computer memory. The authors tested it in a simulation environment called AI Habitat and shared the code publicly.
Autonomous navigationVision-Language ModelVisual servoingChain-of-Thought architectureGrounded-SAMCamera projection geometryZero-shot navigationIndoor roboticsAI HabitatSemantic understanding
Authors
Alperen Avan, Jordi Sanchez-Riera
Abstract
Autonomous indoor navigation requires both semantic understanding and precise geometric control. We propose OptiSight, a hybrid framework that combines Vision-Language Model reasoning with deterministic visual servoing through a finite-state Chain-of-Thought architecture. Grounded-SAM localizes open-vocabulary targets, while camera projection geometry converts visual observations into navigation commands without requiring dense mapping. The VLM is queried only at key decision points, reducing computational overhead while geometric control handles continuous navigation. Experiments in AI Habitat demonstrate reliable zero-shot navigation across diverse indoor scenarios, including obstacle avoidance and semantic ambiguity, while operating within an 8~GB VRAM budget. The source code is available at https://github.com/avanalperen/OptiSight-Python-Multimodal-CoT-for-Visual-Reasoning.