Adding noise to AI weights improves weather forecast uncertainty estimates
Stochastically Perturbed Weights: Ensembles from Deterministic Machine-Learning Weather Models
Machine Learning
Summary
Weather models powered by machine learning often give one single prediction without saying how confident they are. The authors explore a way to produce multiple possible forecasts (an ensemble) by slightly changing the computer model’s settings at prediction time, without retraining it. They find that adding random noise to certain parts of these models can give a good estimate of uncertainty, though it depends on which parts are changed and the model used. This method works fairly well compared to models that are trained specifically to provide such uncertainty but needs careful tuning and has some limitations.
What this means in practice
- •For weather forecast engineers: Use weight perturbation to generate uncertainty estimates from existing AI weather models without retraining, improving forecast reliability at no extra training cost.
- •For energy grid operators: Incorporate improved uncertain weather predictions from AI models into risk assessment for grid management, enabling better planning under weather variability.
Authors
Simon Adamov, Oliver Fuhrer, Reto Knutti, Sebastian Schemm
Abstract
Machine-learning weather models (MLWMs) now match or outperform operational numerical weather prediction (NWP) at global medium-range forecasting, at far lower inference cost. Many deployed MLWMs are deterministic, producing a single forecast with no estimate of its own uncertainty, whereas a growing family of trained-probabilistic models generate calibrated ensembles directly, at the price of a dedicated training run. We ask instead how much uncertainty can be extracted from a deterministic checkpoint that already exists, without retraining it. Where physical ensembles represent model uncertainty by stochastically perturbing parametrisation tendencies, we perturb the network's raw weight tensors at inference time, a scheme we call stochastically perturbed weights (SPW). We also ask whether it works, where and on which scales to inject the noise, and where it fails. A three-phase ablation across four deterministic backbones, Aurora, GraphCast, SFNO, and AIFS, selects one production baseline per model, benchmarked against the trained-probabilistic AIFS-ENS, FourCastNet 3 and Atlas as well as the operational ECMWF ensemble (IFS-ENS) over 112 initialisation times. At a 240 h (10-day) lead time the SPW ensembles reach continuous ranked probability skill scores (CRPSS) between 0.04 and 0.13 below the best trained-probabilistic baseline, at zero marginal training cost. No injection site works across models: the productive tensor group is architecture-specific, so SPW is at present a tuning procedure rather than a plug-and-play recipe. Its main failure mode is a coherent whole-field offset that overdisperses the domain mean, and restricting the noise to coarse scales or perturbing the initial conditions each repair part of it.