Large language model uncertainty measured by internal mechanism stability
Measuring Mechanistic Uncertainty in Large Language Models via Causal Robustness
Computation and LanguageArtificial Intelligence
Summary
Large language models can sometimes give wrong or made-up answers, which is a problem for important tasks. The authors introduce a way to measure how certain a model is by checking how stable its inner workings are when giving a response. They find that when a model is uncertain or wrong, small changes inside can cause its answer to change, showing less stability. Their method adds new insight beyond just looking at the model’s final answer and can help understand when the model really knows something.
What this means in practice
- •For software engineers: Detect when language model outputs are less reliable by measuring internal mechanism sensitivity during question answering.
- •For machine learning operations teams: Monitor model uncertainty changes after training updates to identify if newly learned information is mechanistically stable.
Authors
Aneesh Komanduri, Majid Afshar, Timothy A. Miller, Yanjun Gao
Abstract
Large language models (LLMs) are increasingly used in critical domains, where their susceptibility to hallucinations makes reliable and interpretable uncertainty quantification essential. We introduce the concept of mechanistic uncertainty, which characterizes uncertainty through the stability of the internal mechanisms supporting a model's response. We propose the causal robustness margin, defined as the minimum internal intervention required to deviate the model's output, and operationalize it using Jacobian Lens (Jlens) concept steering vectors. Across three open-source models and five question-answering datasets, we show incorrect and uncertain responses exhibit smaller margins, indicating greater sensitivity to internal interventions, and the causal robustness margin complements conventional output-based uncertainty methods. Finally, in a controlled knowledge injection experiment, correctly learned facts show increased margins after training, suggesting that causal robustness provides a mechanistically grounded signal of uncertainty.