New method improves neural network uncertainty estimates at scale
Scalable AI Uncertainty Quantification via Generalized Laplace Active Subspaces
Artificial Intelligence
Summary
Knowing how uncertain a neural network is about its predictions is important but usually requires too much computing power. The authors present a new way to approximate uncertainty using fewer important directions in the network’s parameter space, making it faster and more practical. They show that adjusting how uncertainty is scaled helps keep predictions reliable and clear, avoiding problems that happen with older methods. This technique can help make AI safer and more trustworthy in real-world settings.
What this means in practice
- •For machine learning engineers: Improve uncertainty estimates in neural networks without heavy computation for safer scientific predictions.
- •For data scientists in finance: Generate calibrated confidence intervals in risk models using neural networks with scalable uncertainty quantification.
Authors
Wouter N. Edeling, Peter V. Coveney
Abstract
Reliable uncertainty quantification (UQ) is essential for deploying neural networks in scientific and high-stakes applications, but full Bayesian inference over the network parameters is computationally infeasible. We propose a low-rank generalized Laplace approximation for neural-network UQ based on a small number of data-informed curvature directions. Starting from a generalized Bayesian posterior defined through an empirical loss, we construct a local Gaussian approximation around a pretrained set of weights in this active curvature subspace. The posterior variances in the retained subspace are available in closed form, and the prior variance is calibrated by an empirical Bayes procedure. The generalized Bayesian formulation allows us to compare two posterior scalings: the standard Bayesian scaling associated with the summed negative log likelihood, and a mean-loss scaling in which the empirical loss is normalized by the number of data. A central finding is that the standard scaling induces a data-size dependent contraction of the posterior variance in the leading active directions. In regression problems, this can force the low-rank framework to retain additional weak-curvature directions in order to achieve nominal coverage of calibration data. When posterior samples are propagated through the non-linear network, these additional directions can degrade the coherence of the predictive intervals and shift the posterior predictive mean away from the pretrained model. In contrast, the generalized mean-loss scaling yields a more stable, lower dimensional active subspace and produces calibrated, coherent predictive confidence intervals. These results indicate that generalized Laplace active subspaces provide a practical and scalable route to calibrated uncertainty quantification in neural networks.