Summary
Forecasting the weather accurately is hard, and knowing how confident we are in a forecast is even harder. Many machine-learning weather models give only one answer without saying how uncertain they are. The authors explore a way to create multiple slightly different forecasts from one model by adding randomness directly to the model’s internal settings without retraining it. This method, called stochastically perturbed weights, works fairly well but needs careful tuning for each model and sometimes produces biased results that need fixing. Overall, it offers a cheaper way to estimate weather forecast uncertainty compared to existing methods that require extra training.
machine-learning weather modelsnumerical weather predictiondeterministic modelsprobabilistic forecastingensemble forecastingmodel uncertaintyinferencestochastic perturbationweight tensorscalibrated ensembles
Authors
Simon Adamov, Oliver Fuhrer, Reto Knutti, Sebastian Schemm
Abstract
Machine-learning weather models (MLWMs) now match or outperform operational numerical weather prediction (NWP) at global medium-range forecasting, at far lower inference cost. Many deployed MLWMs are deterministic, producing a single forecast with no estimate of its own uncertainty, whereas a growing family of trained-probabilistic models generate calibrated ensembles directly, at the price of a dedicated training run. We ask instead how much uncertainty can be extracted from a deterministic checkpoint that already exists, without retraining it. Where physical ensembles represent model uncertainty by stochastically perturbing parametrisation tendencies, we perturb the network's raw weight tensors at inference time, a scheme we call stochastically perturbed weights (SPW). We also ask whether it works, where and on which scales to inject the noise, and where it fails. A three-phase ablation across four deterministic backbones, Aurora, GraphCast, SFNO, and AIFS, selects one production baseline per model, benchmarked against the trained-probabilistic AIFS-ENS, FourCastNet 3 and Atlas as well as the operational ECMWF ensemble (IFS-ENS) over 112 initialisation times. At a 240 h (10-day) lead time the SPW ensembles reach continuous ranked probability skill scores (CRPSS) between 0.04 and 0.13 below the best trained-probabilistic baseline, at zero marginal training cost. No injection site works across models: the productive tensor group is architecture-specific, so SPW is at present a tuning procedure rather than a plug-and-play recipe. Its main failure mode is a coherent whole-field offset that overdisperses the domain mean, and restricting the noise to coarse scales or perturbing the initial conditions each repair part of it.