Focus When Necessary: Adaptive Routing and Collaborative Grounding for Training-Free Visual Grounding
2026-06-15 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionComputation and Language
AI summaryⓘ
The authors present LazyMCoT, a method that helps language models understand detailed parts of complex images without extra training. Instead of treating all images the same, their system smartly decides which images need extra attention based on how uncertain the model is. For tricky images, it uses a special two-step process to zoom in and clarify small or hidden parts. Their tests show it improves accuracy while also running faster on average compared to other methods.
Multimodal Large Language Modelscross-modal reasoningvisual groundingadaptive routingpredictive uncertaintyconformal calibrationtwo-stage refinementimage scalinglocalized cropping
Authors
Yifan Wang, Peiming Li, Shiyu Li, Zhiyuan Hu, Xiaochen Yang, Wenming Yang, Yang Tang, Zheng Wei
Abstract
While Multimodal Large Language Models (MLLMs) excel in cross-modal reasoning, they often struggle to perceive fine-grained details in complex high-resolution images. Recent training-free methods address this through image scaling and localized cropping. However, applying these manipulations indiscriminately introduces computational redundancy for simple queries and can degrade accuracy by truncating essential global context or introducing irrelevant background noise. To this end, we propose LazyMCoT, a dynamic and training-free framework that adaptively allocates visual grounding efforts based on sample difficulty. The framework features an Adaptive Routing mechanism that evaluates predictive uncertainty using first-token statistics from a single forward pass. This efficiently bypasses confident cases while ensuring the recall of difficult samples via conformal calibration. For these challenging cases, a Collaborative Grounding module integrates the inherent cross-modal attention of the model with an external visual expert through a two-stage refinement process. This refinement process generates a precise localized display to recover small or occluded targets. Extensive experiments across diverse benchmarks demonstrate that LazyMCoT rivals training-based approaches by simultaneously improving reasoning accuracy and reducing average inference latency. Our code is availble at https://github.com/TencentBAC/LazyMCoT.