Attention dispersion helps spot errors in large language models
Attention Dispersion as a Diagnostic Signal for Hallucination in Large Language Models
Computation and LanguageMachine Learning
Summary
Large language models sometimes make things up, which can be a problem when they try to solve difficult problems. Traditional ways to detect these mistakes rely on how confident the model looks in its answers, but these are often unreliable. The authors found that looking inside the model at how its attention changes over time can show when it might be guessing or confused. They tested this idea on math problems and found it works better than just checking the model’s confidence. This method could help make AI more trustworthy in the future.
What this means in practice
- •For ai safety engineers: Detect when language models generate unreliable answers by monitoring attention patterns instead of vulnerable output confidence scores.
- •For nlp system developers: Improve hallucination detection in language model reasoning tasks by integrating attention dispersion metrics to enhance model reliability.
Authors
Shardul P. More, Tanuja S. Pawar
Abstract
Large Language Models (LLMs) frequently exhibit hallucinations, presenting a major barrier to reliability in complex reasoning tasks. While traditional detection methods rely on output-based confidence metrics, these logits are often miscalibrated by modern alignment techniques. In this paper, we investigate the temporal volatility of internal attention mechanisms as an alternative diagnostic signal for hallucination that does not depend on output calibration. By introducing an unsupervised metric for attention dispersion, we show that epistemic uncertainty leaves a measurable trace within intermediate layers, where spikes in attention entropy are associated with reasoning breakdowns. We evaluate our approach on mathematical reasoning benchmarks (GSM8K and MATH-500) using the Qwen2.5 model family (1.5B and 3B parameters), finding statistically significant AUC improvements of up to +0.076 over output-based baselines across all tested conditions. These findings suggest that attention dispersion is a promising complement to traditional hallucination detection methods, requiring further investigation across broader model families and task domains.