Multimodal models learn to adapt thinking style and length for tasks
Adaptive Look, Flexible Thought: Learning How and How Long to Think in Multimodal Reasoning
Computer Vision and Pattern RecognitionArtificial IntelligenceComputation and Language
Summary
Multimodal large language models, which understand both pictures and text, usually handle visual and textual reasoning separately. This separation limits how well they can work together on complex tasks. The authors created a method called L2T that treats visual and text reasoning as one process, deciding how much and in what way to think based on the task. Tests show L2T performs as well as current methods but can work 1.7 to 3.6 times faster.
What this means in practice
- •For ai product engineers: Build AI systems that dynamically adjust how they process images and text together for faster and accurate multimodal reasoning.$Commercial implications: Enables development of more efficient multimodal AI tools for industries needing real-time visual and textual data interpretation.
- •For multimedia software developers: Create apps that optimize the balance of visual and textual analysis to improve speed without losing accuracy in complex multimodal tasks.
Authors
Quanlin Chen, Xiuwei Chen, Kun Xiang, Zehua Ma, Chunjin Ye, Boqiang Guo, Xianming Liu, Xiaodan Liang
Abstract
The introduction of long chain-of-thought (CoT) reasoning and external tools has substantially improved the performance of multimodal large language models (MLLMs) on general and reasoning tasks. However, existing fast-slow reasoning paradigms mainly regulate textual reasoning length, while visual computation introduced by external tools or visual tokens is often treated as an auxiliary component. This separation prevents MLLMs from jointly optimizing visual perception and textual reasoning. To address this issue, we propose L2T (adaptive Look, flexible Thought), a unified reasoning framework that jointly optimizes the form and depth of computation. L2T first discretizes tool-produced visual states into visual tokens, bringing visual perception and textual reasoning into a shared autoregressive space. Based on this unified representation, L2T introduces a modality-aware routing reward to select appropriate reasoning modes and a difficulty-aware fast-slow strategy to adapt reasoning depth according to task complexity. Experiments on general perception and reasoning-intensive tasks demonstrate the effectiveness of L2T. Compared with explicit tool-calling approaches, L2T maintains competitive performance while achieving 1.7--3.6 times higher throughput.