Neural pde solvers mimic numerical scheme errors not exact physics
The Symbol of the Surrogate: Measuring Numerical Provenance in Neural PDE Solvers
Machine Learning
Summary
Neural networks meant to predict physical changes often learn the quirks of the computer methods used to generate their training data, not the true underlying physics. The authors show that evaluating these networks on similar data doesn't reveal this problem. They introduce a technique using Fourier analysis to test whether the networks've learned the actual physics or just the numerical shortcuts. Their tests reveal the networks mostly copy the numerical solver's behavior, not the exact physical evolution.
What this means in practice
- •For numerical software developers: Detect when learned PDE models inherit solver-specific errors, enabling more reliable surrogate model assessment.
- •For climate modelers: Identify and reduce error sources in neural surrogates used for weather and climate simulations by diagnosing numerical scheme imprinting.
Authors
Ridham Patel
Abstract
Neural PDE surrogates are trained on numerical solver outputs that contain both physical evolution and solver-specific discretization errors. Because surrogates are also evaluated against held-out trajectories from the same solver, standard benchmarks cannot distinguish fidelity to the exact evolution from imitation of the numerical scheme. We introduce an empirical Fourier-symbol diagnostic that probes a trained surrogate's linearized one-step operator with individual Fourier modes and compares it with both exact-evolution and training-scheme references. To address architectural spectral bias, we train identical networks on schemes with orthogonal dissipative and dispersive signatures and compare their learned operators. In linear advection, the learned surrogates reproduce the training schemes' amplitude and phase errors, with the twin-scheme difference reaching more than 99.8\% of the analytically predicted full-imitation ceiling. The same behavior occurs for a non-local Fourier neural operator and at the operator level for nonlinear Burgers dynamics. These results show that agreement with solver-generated test data does not by itself establish fidelity to the exact evolution. Fourier-symbol measurements provide a direct diagnostic of numerical provenance.