Neural solvers improve consistency for partial differential equations
Same Target, Different Origins: Aligning Latent Rollout for Consistent PDE Emulation
Machine Learning
Summary
Predicting how things change over time using computers can get less accurate when the model repeats its own outputs as new inputs, causing small errors to build up. The authors created a method called Align Latent Rollout (ALR) that helps the model keep its internal predictions consistent no matter when or how it starts predicting. This means the model better matches the true behavior of the system it’s simulating, leading to more accurate and reliable forecasts. They tested this on different equation benchmarks and showed that ALR reduces errors and discrepancies between predictions made starting from different times.
What this means in practice
- •For simulation engineers: Enhance multi-step forecasts by neural models to produce consistent, accurate simulations of complex physical systems governed by PDEs.
- •For software developers in scientific computing: Integrate ALR regularization into existing neural PDE solver software to improve reliability of long-term predictions without changing core architectures.
Authors
Xinying Li, Jing Xu, Wenjun Wang, Limei Han, Fenglei Cao, Ruoxi Jiang, Yuan Qi
Abstract
Neural PDE solvers commonly generate multi-step predictions through autoregressive rollout, recursively applying a learned evolution operator and feeding each prediction back as input for the next step. Approximation errors can therefore propagate through the rollout, causing predictions at a shared target time to depend on their temporal origins and rollout depths. For a well-posed deterministic PDE flow, however, states at different times along the same physical trajectory should, under exact evolution, reach the same target state. To this end, we propose Align Latent Rollout (ALR), a regularization approach that promotes consistency across rollout origins through the neural PDE solver's internal representations. A second-order Taylor analysis of latent discrepancy propagation motivates two objectives: zeroth-order latent state consistency, which aligns representations at a shared target time, and first-order latent dynamics consistency, which aligns local evolution responses to shared perturbations. ALR can be integrated into different neural PDE solvers by regularizing suitable internal representations, without modifying their backbone architectures. Experiments across multiple PDE benchmarks and neural-operator backbones show that ALR improves prediction accuracy while reducing shared-target discrepancies across rollout origins.