Metric bench improves visual spatial reasoning in AI models indoors
Metric-Bench: Exploring In-context Spatial Metric Reasoning in VLMs for Indoor Scenes
Computer Vision and Pattern RecognitionMultimedia
Summary
Understanding distances and sizes inside buildings is hard for AI models that see and read. The authors created Metric-Bench, a test that helps AI learn how to figure out real-world sizes using known objects in images, without extra camera details. They also made MetricReasoner, a training method that improves this skill by giving clear feedback. Their approach made AI better at understanding space and size, even beating bigger models and helping robots navigate and act indoors.
What this means in practice
- •For robotics developers: Improve indoor navigation and manipulation by training robots to better estimate object sizes and distances using visual cues.
- •For augmented reality app creators: Enhance AR apps to more accurately gauge spatial layouts indoors without calibration by referencing known object sizes in the scene.
Authors
Yuling Xi, Haokai Zhang, Muzhi Zhu, Hao Zhong, Zongze Du, Hengyu Zhao, Chenchen Jing, Yufei Yin, Bin Qin, Yongjie Yang, Zhenbo Luo, Hao Chen, Chunhua Shen
Abstract
Metric reasoning is a critical and challenging task for Vision Language Models (VLMs), playing a pivotal role in embodied AI tasks such as robotic manipulation and autonomous navigation. However, current spatial reasoning remains bottlenecked by rigid pixel-level supervision; such localized optimization often compromises general multimodal intelligence, triggering performance degradation or catastrophic forgetting of broad reasoning capabilities. To address these limitations, we introduce Metric-Bench, a focused benchmark designed to guide metric-spatial reasoning using contextual information. By incorporating in-image reference objects with known physical dimensions, Metric-Bench guides models to implicitly learn the 2D-to-3D mapping without camera intrinsics. We further present MetricReasoner, a task-adapted reinforcement fine-tuning recipe for reference-grounded metric reasoning, using structured prompts and verifiable numerical rewards. Extensive experiments on Metric-Bench demonstrate that our approach significantly enhances spatial metric understanding, outperforming existing and even larger proprietary models by 43.1\%, while improving downstream embodied performance over a spatial-specialized counterpart by 30.4\% on RoboSpatial overall accuracy and 9.3\% on ERQA, and additionally delivering consistent gains on general benchmarks (15.9\% on V$\star$Bench, 88.9\% on BLINK), indicating that the proposed adaptation does not necessarily compromise general VLM capabilities.