Checkable conditions ensure reliable training for physics informed neural networks

Checkable NTK Positivity and Finite-Width Gradient Descent for Scalar- and Vector-Valued PINNs with Strong-Form, Weak-Form, and Nonlocal Linear Constraints

Machine Learning

Summary

Physics informed neural networks (PINNs) are used to solve physics problems by training artificial neural networks with physical rules as constraints. This paper provides a way to check before training whether the neural network's design will learn well by examining a mathematical kernel called the neural tangent kernel (NTK). The authors show that if certain conditions are met, gradient descent training will reliably reduce errors even when the network is not infinitely large. This helps ensure the network can accurately model physics under various kinds of constraints such as differential equations, boundary conditions, or integral observations.

What this means in practice

Authors

Zifan Lyu

Abstract

We give checkable positive-definiteness criteria for the limiting neural tangent kernel (NTK) and high-probability finite-width gradient-descent guarantees for scalar- or vector-valued physics-informed neural networks (PINNs) with linear constraints. The constraints may be strong-form differential rows of any fixed finite order, including coupled systems, with any linear initial or boundary conditions; weak-form residual and boundary functionals; or finite-measure nonlocal observations such as integral, nonlocal-diffusion, and Dirac-data rows, all in any dimension. For each class, positive definiteness of the limiting NTK is equivalent to a rank condition on a finite coefficient, functional, or moment matrix of the fixed design: a certificate computed before training that detects structural zero modes. The model hypotheses are those of an ordinary two-layer PINN, a smooth nonpolynomial activation with bounded symmetric initialization, and are met by standard choices such as $\tanh$ with uniform initialization. Given a certificate, explicit width and step-size conditions ensure, with high probability, that the empirical NTK retains at least half the limiting gap and that full-batch gradient descent decreases the training loss geometrically at every iteration. Controlled experiments check the certificates and the finite-width mechanism.