Pde solvers benefit from machine learning trained on smaller problems
Warm-starting PDE solvers with any-dimensional machine learning
Machine Learning
Summary
Solving complicated equations that describe things like heat or fluid flow in many dimensions usually takes a lot of computer time. The authors show that machine learning models trained on simpler, lower-dimensional examples can help start solutions for more complex, higher-dimensional problems without retraining. This approach saves computing resources and still improves accuracy. They tested it on common physics equations like the heat equation and Navier-Stokes and found better results with less data and computation.
What this means in practice
- •For computational fluid dynamics engineers: Speed up high-dimensional fluid simulation by warm-starting solvers with models trained on lower-dimensional data.
- •For software developers for physics simulation: Reduce training data and computing power for PDE solvers by leveraging transfer learning from smaller-scale problems.
Authors
Wilson G. Gregory, George A. Kevrekidis, Ben Blum-Smith, Soledad Villar
Abstract
Any-dimensional machine learning models, such as graph neural networks (GNNs), can be naturally trained and evaluated on inputs of different sizes and dimensions. Inspired by the GNN transferability literature, we show mathematical conditions under which a partial differential equation (PDE) learning-based solver can be trained in small dimensions and directly applied to solve a higher dimensional PDE in a zero-shot fashion. These conditions are based on symmetries in both the partial differential equation and the initial data. When the equations satisfy the symmetries but the data does not, which is the case for many PDEs arising from physics, we show that our theory gives a principled way of warm-starting low-dimensional PDE solvers for higher dimensional PDEs. We apply this method on the heat equation, Burgers' equation, and the compressible Navier--Stokes equations, improving the performance in both zero-shot and typical training regimes on high dimensional data. For example, we train a surrogate model on 2D Navier--Stokes data and achieve better results on 3D test data than a baseline surrogate model trained on 3D data, while only using 12$\%$ of the flops and 20$\%$ of the total data size.