TangentPDE improves neural PDE operators with local sensitivity training
TangentPDE: Learning Local Operator Responses with Sparse Finite-Difference Supervision
Machine Learning
Summary
Solving partial differential equations (PDEs) with neural networks usually focuses on matching the final solution but overlooks how the solution changes when inputs are slightly altered. The authors present TangentPDE, a method that teaches neural models to better understand these local changes by comparing how small input tweaks affect outputs using finite differences. This approach doesn’t need complex derivative calculations and works with nonlinear PDE solvers. Their tests show TangentPDE reduces errors in many cases by learning both the solution and its local variations more accurately.
What this means in practice
- •For computational physics engineers: Enable more reliable simulations by improving neural models’ understanding of how small changes affect complex PDE-based systems.
- •For machine learning platform developers: Build training frameworks that integrate sensitivity learning to boost performance of neural operators across diverse PDE tasks.
Authors
Xinjian Xie, Ruxuan Chen, Jiashu Li, Tianyu Gao, Ye Wei
Abstract
Neural PDE operators are often optimized through state reconstruction, which largely constrains operator values at observed inputs but does not fully characterize their sensitivity to local perturbations. A more faithful approximation of operator behavior requires not only accurate state predictions, but also correct variations along local perturbation directions. We present TangentPDE, which uses finite differences to approximate directional derivatives and constructs task-relevant finite-step responses from the state outputs of a reference solver at sparse paired inputs. Its objective jointly constrains state fidelity, response-direction alignment, and response-magnitude calibration, thereby learning both operator values and their local variations. Because the response labels are constructed solely from state outputs at paired inputs, the method requires neither explicit derivative labels, linearized or adjoint PDE solves, nor full Jacobians, and remains applicable to nonlinear solvers without derivative interfaces. Building on this, we additionally propose a selectively executed adaptation mechanism to strengthen local operator representations under finite-difference supervision. Budget-matched and correspondence-perturbed controls isolate the effect of relational supervision from improvements attributable to additional state observations or increased data volume. Designed as a plug-and-play training framework, TangentPDE integrates with diverse neural-operator backbones. Across four PDE systems and 20 task-backbone settings, TangentPDE lowers the three-seed mean state error in 18 settings and the corresponding Rollout@20 error in 18, while intrinsic finite-step response gains are strongest on DiffReact and remain task- and backbone-dependent elsewhere.