Hard constraints shape climate emulators beyond forecast corrections
Evaluating Predictor-Corrector Dependence in Hard-Constrained Climate Emulators
Computers and SocietyComputational Engineering, Finance, and Science
Summary
Climate emulators use neural networks to predict future weather but often adjust the predictions to make sure they follow physical laws. The authors show that simply looking at the final corrected forecasts hides how these adjustments also change the neural network’s own predictions underneath. They developed a way to test the predictions before and after correction and found that training with corrections changes the model’s behavior in subtle ways, affecting forecast accuracy at different timescales. This means it’s important to evaluate both the raw predictions and the corrections separately to understand how these models really work.
What this means in practice
- •For weather model developers: Design evaluation strategies that measure both uncorrected predictions and correction effects to better understand model accuracy and physical consistency.
- •For climate simulation engineers: Improve training techniques by incorporating correction effects separately to enhance long-term forecast accuracy in climate emulators.
Authors
Tianyu Zhang, Xinyu Wang, Zhenghan Tai, Xiao-Wen Chang, Yoshua Bengio
Abstract
Climate emulators often enforce physical constraints by correcting each state predicted by a neural network, but evaluating only the corrected forecast conflates predictor skill with the effect of the correction. We introduce an evaluation framework that separates these contributions by comparing corrected forecasts, pre-correction predictions along corrected trajectories, and autonomous rollouts with correction disabled. We further cross correction during training and inference to distinguish its immediate effect from how it shapes the learned predictor, and use matched nonphysical controls to test whether improvements are specific to the physical correction. In ACE2, correction substantially improves physical consistency. In a three-seed ACE2 pilot, models trained with correction produce less accurate uncorrected precipitation forecasts at short horizons than models trained without correction, but more accurate forecasts at 10 and 30 days. This contradicts our preregistered hypothesis that the deficit would grow with forecast horizon. In a separate nine-seed experiment, directly supervising the uncorrected output reverses its precipitation forecast deficit at 72 hours while maintaining the accuracy of corrected forecasts at 24 hours. In ConCerNet's heat-equation system, projection-trained models have higher uncorrected trajectory-based vector-field RMSE than models trained without projection, while restoring the physical projection recovers accuracy far more effectively than a matched nonphysical control. Together, these results show that hard corrections can reshape the predictors trained alongside them, creating dependencies that are not visible from corrected forecasts alone. Hard-constrained emulators should therefore be evaluated by separately measuring predictor accuracy, correction effects, and their interaction during training.