DICA: Dual-Indicator Guided Contrastive Alignment in Multimodal Large Language Models
2026-07-27 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors studied how humans look at images carefully, starting with the whole picture and then zooming in on important parts, but many AI models don't do this well and sometimes make stuff up. They created a new method called Dual-Indicator Guided Contrastive Alignment (DICA) that watches two signs during AI thinking: how spread out the AI's image attention is (VAE) and how much the AI's answer depends on the image (OIC). When these signs show something is off, DICA fixes the model's focus to make better, more grounded answers. Tests show DICA works better than older methods at reducing made-up details in AI responses.
Visual reasoningMultimodal large language modelsAttention entropyContrastive alignmentVisual groundingHallucinationsInferenceVisual attentionImage correlationInformation-theoretic indicators
Authors
Hao Yang, Jin Wang, Xuejie Zhang
Abstract
Human visual reasoning typically follows a coarse-to-fine attention process, starting from global scene understanding and gradually focusing on question-relevant regions. However, multimodal large language models may deviate from this pattern due to attention drift and the underutilization of visual evidence, which can lead to hallucinations. To mitigate these issues, this study proposes a Dual-Indicator Guided Contrastive Alignment (DICA), which tracks two information-theoretic indicators during inference: Visual Attention Entropy (VAE), which reflects the concentration of visual attention, and Output Image Correlation (OIC), which measures the dependence of generated outputs on the visual input. An abnormal increase in VAE or a decrease in OIC corresponds to different failure modes, which trigger targeted contrastive alignment to restore visual grounding. Experimental results across multiple benchmarks demonstrate that DICA consistently outperforms existing approaches and substantially reduces hallucinations, highlighting the effectiveness of indicator-driven intervention in improving multimodal inference reliability. The code is publicly available at https://github.com/BGWH123/DICA/.