Physics informed networks improve 3d equation solving with conflict free gradients
Tackling Failure Modes of PINNs and PIKANs Using Conflict-Free Gradients
Machine Learning
Summary
Solving complex math problems that describe physical systems often involves breaking the problem into parts and training AI models on each part. But training can be tricky because different parts pull the model’s learning in conflicting directions. The authors studied existing methods to avoid these conflicts and found they sometimes fail for complicated 3D problems. They created a new technique called Norm-PCGrad that fixes these conflicts more reliably and improves accuracy without slowing down training. Their approach also works well on different AI models and speeds up computation by simplifying parts of the network.
What this means in practice
- •For computational engineers: Enhance numerical solvers for complex physical systems by using Norm-PCGrad to reduce training conflicts in multi-domain physics-informed neural networks.
- •For software developers in scientific computing: Implement efficient domain decomposition methods that lower computation costs via separable architectures like SPINN within physics-informed network frameworks.
Authors
Sidharth S. Menon, Irina Tezaur, Ameya D. Jagtap
Abstract
Scientific machine learning methods such as physics-informed neural networks (PINNs) increasingly rely on domain decomposition for better scalability while solving partial differential equations (PDEs) over complex geometries, yet the resulting composite loss comprising residual, boundary, and interface terms is highly susceptible to conflicting gradients that degrade training. This work bridges domain decomposition with projection-based gradient surgery to systematically mitigate such conflicts in 2D and 3D settings. We evaluate two existing projection-based algorithms, PCGrad and ConFIG, and identify their performance degradation in specific scenarios such as 3D domains with multiple overlapping interfaces. To address this limitation, we propose Norm-PCGrad, a normalized variant that achieves state-of-the-art accuracy across a range of 2D and 3D domain decomposition problems. Across the benchmarks considered, Norm-PCGrad consistently achieves the lowest relative $L_2$ error compared to training without gradient surgery as well as to existing algorithms such as PCGrad and ConFIG, while incurring negligible additional computational overhead. To improve computational efficiency of domain decomposition frameworks such as Extended PINN (XPINN), we propose replacing vanilla PINNs in selected subdomains with separable architectures such as Separable PINN (SPINN), reducing the computational cost from quadratic (or cubic) to linear. We additionally demonstrate that gradient surgery extends to physics-informed Kolmogorov-Arnold Networks (PIKANs), yielding substantial accuracy improvements for 3D domain decomposition and confirming the generality of the proposed approach across network architectures.