Latent Noise Mask for Reducing Visual Redundancy in Multimodal Large Language Models
2026-06-29 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors found that big language models that also look at pictures often struggle when they need to reason about small details in images because important visual information gets mixed up with too much irrelevant stuff. To fix this, they created a method called Lens that helps the model focus on the parts of the image that matter for the question. Lens works by giving scores to image parts and gently hiding irrelevant sections during processing, without changing the main model or image input. This approach improves the model's accuracy on various visual question-answering and grounding tests using a simple addition to the model.
Multimodal Large Language ModelsVisual Question Answering (VQA)Latent SpaceChain-of-ThoughtVisual TokensLatent NoiseToken ScoringGrounding TasksVisual ReasoningNoise Injection
Authors
Kai Jiang, Ruishu Zhu, Siqi Huang, Hongyuan Zhang, Xuelong Li
Abstract
Multimodal large language models (MLLMs) often fail in fine-grained visual reasoning, as question-relevant visual cues are diluted by dense and redundant image tokens. Recent multimodal reasoning methods usually extend chain-of-thought from language models into visual or latent spaces, seeking to add intermediate reasoning states while overlooking the negative impact of redundant visual tokens. We propose LatEnt Noise maSk (Lens), a question-conditioned visual evidence purification framework that empowers MLLMs to reason with cleaner visual cues in latent space. Lens introduces a lightweight Lens Evidence Token (LET) to score which visual tokens support the current question and preserve them during decoding. Guided by the LET scores, it injects adaptive latent noise into low-relevance tokens, softly suppressing distractors without changing the model backbone or token sequence. With only one temporary learnable control token and a lightweight noise generator, Lens adds minimal overhead while improving the base MLLM by 2.4-6.4 points on most VQA datasets and by 4.1-6.4 points on grounding tasks. These results show that multimodal reasoning can benefit more directly from cleaner question-relevant visual evidence than from simply extending the reasoning trace.