First-layer gating improves physics-informed neural network accuracy
LPINNs: First-Layer Gated Localization for Physics-Informed Neural Networks
Machine LearningComputational Engineering, Finance, and ScienceNeural and Evolutionary Computing
Summary
Physics-informed neural networks (PINNs) often struggle to learn solutions for problems defined over large spaces or involving complex equations. The authors show that modifying only the first layer of a neural network by multiplying activations with specific input-based localization functions helps the network focus on different parts of the problem domain separately. By testing many types of these localization functions on different physics problems, they found some that consistently reduce errors significantly compared to standard PINNs. However, not all localization functions worked well, showing this is a nuanced improvement rather than an automatic fix.
What this means in practice
- •For scientific computing teams: Improve accuracy of PINN models solving complex physics problems on large or high-order domains by integrating input-dependent gating in the first layer.
- •For engineering simulation groups: Reduce error in physics-based simulations using neural networks by applying first-layer localization methods tailored to specific equations and domain sizes.
Authors
Lakshay Chawla, Hardik Jain
Abstract
Physics-informed neural networks (PINNs) use one shared representation over the computational domain, which can become difficult to optimize on long domains and for high-order operators. We study a minimal alternative: multiply the first hidden activation of an otherwise unchanged dense PINN by input-dependent localization functions, giving first-layer units receptive fields without partitioning the domain or adding interface losses. We screen 13 families of localization functions, in up to three parameterizations each, on a nonlinear harmonic oscillator (HO), a heat equation on a long spatial interval, and a manufactured four-dimensional (4D) fourth-order problem, with ten paired seeds throughout. Three configurations give large reductions in solution error at matched budgets: (i) Fixed Gaussian localization functions on the $2π$ HO domain cut mean solution RMSE from $4.8369\times10^{-1}$ to $8.83\times10^{-3}$ at 3k epochs. (ii) The inverse-quadratic family with learnable centers and widths cuts it from $2.896\times10^{-1}$ to $3.06\times10^{-2}$ on the $8π$ heat domain at 10k epochs. (iii) Fixed bump localization functions cut it from $1.75947\times10^{1}$ to $2.260\times10^{-1}$ on the $4π$ 4D domain at 10k epochs. Every paired seed improves in these three comparisons. The screen also shows that the mechanism is not a free win: on HO only 2 of 13 families beat the baseline, and 10 of the remaining 11 are 9 to 23 times worse; on 4D four families are non-finite and five are more than three orders of magnitude worse than the baseline. The inverse-quadratic family is the only one that beats the baseline on all three equations. Overall, these results show that first-layer localization can provide measurable improvements to baseline PINNs on long-domain and high-order problems.