Evaluating Epistemic Uncertainty: Beyond OOD Detection and Active Learning
Machine LearningArtificial Intelligence
Summary
The authors look at how to better measure epistemic uncertainty, which is the uncertainty in what the model doesn't know. They show that the usual ways of scoring this uncertainty don't always match the best decision-making strategies. They propose a new method that combines different types of uncertainty and focus on how well these measures help reduce mistakes. By testing on real data, they find that popular evaluation methods might not reflect how useful the uncertainty estimates really are for making decisions.
Authors
Jakub Paplhám, Willem Waegeman, Eyke Hüllermeier, Vojtěch Franc
Abstract
Current evaluation of epistemic uncertainty relies on tasks such as out-ofdistribution detection and active learning. However, the Bayes-optimal decision strategies for these tasks do not coincide with the scores commonly used to quantify epistemic uncertainty. Building on the epistemic reject-option framework, we evaluate epistemic uncertainty using its ability to identify regret, the reducible error. Formulating selective prediction as a constrained optimization over coverage, expected risk, and regret, we prove the optimal selector is a thresholded convex combination of the ground-truth aleatoric and epistemic uncertainties. This theoretical unification exposes a weakness in recent uncertainty disentanglement literature: we demonstrate that standard correlation metrics between learned components do not necessarily predict their actual operational utility. We instead propose to evaluate the achievable risk, regret, coverage surface of the decomposition as a diagnostic for joint disentanglement and utility. Benchmarking standard methods on datasets with dense human annotations reveals that decision-theoretic rankings can disagree substantially with proxy-task rankings, including pairwise rank inversions between methods that are top-ranked on one criterion and bottom-ranked on other.