Large vision language models reveal when they ignore images during text generation
What Do Hallucinations Reveal About Multimodal Reasoning? Diagnosing Visual Grounding Failures via Contrastive Decoding Probes
Computer Vision and Pattern RecognitionMultimedia
Summary
Visual language models sometimes imagine things that aren't really in the picture when describing images. The authors created a method called SAFE that compares what the model says when it sees the image versus when it doesn't. This helps spot exactly which words are based on the picture versus guesses. They found that mistakes happen in bursts and catching them early can help reduce errors without hurting the flow of language. Their approach works well on a special test set but shows mixed results on others.
What this means in practice
- •For ai developers: Identify and reduce image-irrelevant words during model text generation to improve visual accuracy in multimodal AI systems.
- •For content moderation teams: Detect generated text components that are not supported by image content to flag potential misinformation or errors.
Authors
Zhipeng Zhao, Wenxu Wang, Peishun Liu, Ruichun Tang
Abstract
When strong multimodal models are widely available, progress requires new scientific methodologies beyond benchmark scores---using models as instruments for understanding behavior. We address this by asking: can we use large vision-language models (LVLMs) as experimental instruments for studying their own failure dynamics? Focusing on visual hallucination, we introduce SAFE, a training-free decoding framework that contrasts visually-grounded and vision-ablated generation paths to produce a token-level contrastive grounding score that identifies when the model favors linguistic priors over visual evidence. This signal serves dual roles: as a practical proxy for detecting visually-ungrounded tokens, and as the basis for decoding-time penalties. Our analysis yields three empirical observations: visual dependency decays over generation, hallucinations co-occur in temporal clusters, and early intervention reduces clustering without substantially degrading fluency. On MMHalBench, SAFE substantially outperforms all compared baselines; results elsewhere are more mixed. We argue that designing contrastive probes exemplifies a broader mission: using models as instruments for scientific understanding. Code: https://github.com/zhaozhipeng1997/SAFE_public.