Optimal transport dropout improves neural network uncertainty predictions
Optimal Transport Dropout for Structured Predictive Uncertainty
Machine Learning
Summary
Many neural networks give predictions but don’t say how sure they are about those predictions. The authors introduce Optimal Transport Dropout (OTD), a method that learns how to better capture uncertainty by shaping random changes within the network in a smarter way. Unlike existing methods that treat these changes independently, OTD models them together to reflect more realistic possibilities. This leads to more accurate predictions and better confidence estimates, especially when models have limited data or are working on complex tasks.
What this means in practice
- •For machine learning engineers: Improve regression model reliability by producing well-calibrated uncertainty estimates without multiple trained models or explicit Bayesian inference.
- •For computational physicists: Obtain uncertainty-aware surrogate models for partial differential equations that align predictive confidence with spatial error patterns.
Authors
Giacomo Lorenzon, Francesco Regazzoni
Abstract
Deterministic neural networks and neural operators provide point predictions with no intrinsic measure of reliability. Yet, predictive uncertainty may stem from irreducible outcome variability, finite data, or limitations of the chosen model class. Monte Carlo dropout offers a computationally convenient way to construct a predictive distribution through stochastic feature masking, without training multiple independent networks or explicitly inferring a posterior over model parameters. However, its perturbation law is largely prescribed a priori and typically factorised across latent coordinates. We introduce Optimal Transport Dropout (OTD), which instead learns the predictive mapping and the law of its latent perturbations jointly. Starting from a simple independent reference distribution, OTD transports latent perturbations through a learnable flow and propagates them through the predictive neural network, thereby inducing a structured predictive law. Training uses the strictly proper Energy Score, while a kinetic-action term geometrically regularises the transport. Synthetic benchmarks show that OTD captures multimodal predictive distributions, generates meaningful dispersion when the model is misspecified, and exhibits contracting dispersion as more training data or greater model capacity are provided. For a field-valued partial differential equation surrogate, predictive dispersion strongly aligns with the spatial pattern of prediction errors. On this task, compared with Monte Carlo dropout, OTD yields more accurate predictions and better-calibrated, substantially narrower intervals. On real-world regression benchmarks, it further shows competitive accuracy and better probabilistic predictions compared to established baselines. OTD therefore offers a way to learn structured predictive uncertainty without explicit posterior inference or ensembles of independently trained predictors.