Large language models uncertainty measured with two level ensembles

Black-Box Uncertainty Quantification for Large Language Models via Ensemble-of-Ensembles

Machine LearningArtificial Intelligence

Summary

It is hard to tell if a large language model is unsure because a question is unclear or because it doesn’t have the knowledge. The authors created a way to measure these two types of uncertainty separately by looking at different model answers and slightly changed versions of the question. This method only needs to observe the model’s outputs, without accessing its internal details. Tests on various question-answer tasks show the new method works well and can identify specific reasons for mistakes.

What this means in practice

  • For llm developers: Estimate when models are unsure due to ambiguous prompts versus missing knowledge using only output samples without modifying model internals.
  • For content moderation teams: Identify and handle uncertainties in model-generated answers to improve reliability and trustworthiness in automated response systems.

Authors

Wang Ma, Debarun Bhattacharjya, Junkyu Lee, Nhan H Pham, Harsha Kokel, Qiang Ji

Abstract

Reliable use of large language models (LLMs) requires uncertainty estimates that separate ambiguity intrinsic to a prompt from genuine knowledge gaps. Bayesian and deep-ensemble methods deliver this aleatoric/epistemic decomposition in principle, but are computationally prohibitive at LLM scale and require white-box access. Existing black-box methods, in contrast, summarize to a single scalar that cannot make the distinction. We introduce a two-level ensemble that bridges this gap. Stochastic decoding of a fixed prompt forms the inner ensemble; meaning-preserving semantic perturbations of the prompt form the outer ensemble. Variability is measured in a continuous embedding space, and the law of total covariance gives an exact within-/between-perturbation decomposition to estimate aleatoric and epistemic uncertainty (AU/EU); it requires only black-box sampling access. On top of this estimator, we contribute three theoretical results that ground the framework: (i) a finite-sample bias identity for nested Monte Carlo with a closed-form bias-corrected epistemic estimator; (ii) a Bayesian bridge bounding the gap between perturbation-population AU/EU and Bayesian targets; and (iii) a separation result identifying a class of failure modes that any consistency-only black-box method provably misses while perturbation-based EU detects. Across five short QA benchmarks and four instruction-tuned LLMs spanning two families and two scales, our estimator is competitive with strong white-box baselines, and at a matched generation budget it matches leading black-box methods on error ranking while decomposing AU and EU; controlled paired interventions show that AU and EU respond selectively to distinct, actionable sources of failure: prompt ambiguity and model knowledge gaps; and a long-form pilot on the NQ long-answer split shows the framework extends to claim-level factuality without retraining.