Variational latent models improve long horizon pde predictions
Stable by Construction: Variational Latent Markov Operators for Long-Horizon PDE Prediction
Machine Learning
Summary
Predicting how physical systems change over time using neural networks is hard because small errors add up. The authors developed a method that represents the system in a way that focuses on probabilities and smooth transitions, helping avoid growing mistakes. Their approach models the system’s hidden states with special math tools called variational latent distributions and Gaussian processes. This helps their predictions for things like fluid flow stay accurate even when looking far into the future.
What this means in practice
- •For fluid dynamics engineers: Generate more stable, accurate long-term fluid flow simulations beyond training conditions using variational latent modeling.
- •For climate modelers: Create improved long-horizon models of physical systems governed by PDEs with reduced prediction error growth.
Authors
Junyi Liao, Johann Guilleminot, Vahid Tarokh
Abstract
Neural PDE solvers provide efficient surrogates for time-dependent physical systems, but autoregressive prediction over long horizons remains challenging because local errors can induce distribution shift and accumulate under recursive deployment. We develop a variational approach to this problem by introducing latent Markov dynamics in which physical states are represented by latent distributions and evolved through probabilistic transitions. The framework is formulated directly on function spaces and specialized to functional Gaussian models, where structured latent perturbations induce a spectral geometry and variational transition alignment regularizes the learned dynamics. We further analyze how these mechanisms affect autoregressive error propagation, providing a theoretical connection between variational training and long-horizon prediction. We instantiate the framework as the Variational Autoencoding Markov Operator (VAMO), which combines spatially resolved latent fields, structured Gaussian perturbations, and a neural-operator transition. Empirically, we demonstrate the effectiveness of VAMO on several fluid-dynamics benchmarks with prediction horizons extending substantially beyond those represented during training, where it consistently reduces error accumulation and improves rollout stability over several deterministic and noise-injection baselines. Overall, these results highlight variational modeling as a complementary approach to robust long-horizon neural PDE dynamics.