Neural networks that learn time linked equations more accurately

CEENs: Causality-enforced evolutional networks for solving time-dependent partial differential equations

Machine Learning

Summary

Partial differential equations (PDEs) describe many natural and physical processes but are hard to solve over long times accurately with usual neural networks called physics-informed neural networks (PINNs). The authors show this problem happens because PINNs don't properly consider that causes must happen before effects in time. They introduce a new approach that breaks the time into separate chunks, assigns each chunk its own neural network, and trains these networks in time order to respect this cause-effect sequence. This method improves accuracy significantly for long-term simulations and uses less computing power than traditional PINNs.

What this means in practice

Authors

Jeahan Jung, Heechang Kim, Hyomin Shin, Minseok Choi

Abstract

Despite the growing popularity of physics-informed neural networks (PINNs), their applicability in the long-time integration of partial differential equations (PDEs) remains constrained. We argue that this problem stems from the lack of consideration of temporal causality in the original PINN formulation, resulting in a bias towards satisfying governing equations at later times before learning the initial condition and hence leading to erroneous solutions. To this end, we propose a novel method that seamlessly integrates temporal causality into the training process. Drawing inspiration from classical numerical methods where the temporal causality is reflected, we divide the time domain into nonoverlapping subintervals, assign a unique neural network to each subinterval, and construct a loss function founded on the integral form of PDEs within these subintervals. The proposed networks undergo sequential training, beginning with the initial time step. Our method demonstrates significant improvement in accuracy for long-time simulations of various PDE problems where the original PINN method fails while it requires less computational cost and memory compared to the PINN method. A parallelization algorithm is provided to further enhance the computational efficiency, showing a significant speedup for solving time-dependent PDEs.
View PDFOpen arXiv