PDE foundation models show growing prediction errors over long simulations
Beyond Local Accuracy: Benchmarking the Rollout Reliability of PDE Foundation Models
Machine Learning
Summary
Predicting the future behavior of complex systems using computer models often looks accurate for a short time, but errors can grow when those predictions are used repeatedly to predict further ahead. The authors study different models used for partial differential equations, which describe many physical processes, to see how prediction errors multiply over longer times. They find that models that do well in short tests often have twice as much error when running simulations longer. This means simply measuring short-term accuracy is not enough to tell whether a model will be reliable for long predictions.
What this means in practice
- •For computational engineers: Identify reliable PDE models for long-term simulations by using horizon-resolved error metrics and amplification statistics.
- •For climate model developers: Evaluate climate simulation components for error growth over extended timeframes beyond initial forecast accuracy.
Authors
Jaewan Park, Seid Koric, Giri Krishnan, Sudhakar Pamidighantam, Surresh Marru
Abstract
PDE foundation models enable data-efficient adaptation but are deployed autoregressively, so short-horizon accuracy need not imply reliable simulation. We ask whether the short-horizon advantage of a foundation-model recipe persists when its predictions are repeatedly fed back as inputs for a long horizon. We define error amplification as G=E(H_f)/E(H_s), which measures full-horizon error relative to short-horizon error. We evaluate seven models on seven two-dimensional time-dependent PDE datasets: scratch-trained FNO and DeepONet; fine-tuned DPOT, GPhyT, Poseidon, and Walrus; and frozen in-context VICON. We report VRMSE, relative L_2, and relative MAE. Six datasets provide paired short- and full-horizon scores, while CE-RM contributes short-rollout sweeps only. At least one fine-tuned foundation model outperforms both scratch baselines in 26 of 30 short-rollout dataset--budget cells under every metric. Nevertheless, full-horizon VRMSE exceeds short-horizon VRMSE in 140 of 144 paired trained configurations, with a median error amplification of $2.04$. The count below VRMSE one falls from 104 to 53, and 12 full rollouts exceed ten although none does at the short horizon. Walrus achieves the lowest full-horizon VRMSE in 16 of 24 cells, yet its median error amplification is $2.70$. Relative L_2 and relative MAE show the same qualitative pattern. These results show that favorable short-horizon rankings can coexist with substantial error amplification. PDE foundation-model benchmarks should therefore report horizon-resolved absolute errors, amplification statistics, and failure incidence rather than a single local forecast score.