Automatic differentiation can fail to capture physics constraints

The limits of exactness: On the failure of automatic differentiation in physics-informed machine learning

Machine Learning

Summary

Automatic differentiation (AD) helps computers find exact mathematical derivatives in machine learning models, which is very useful for solving physics problems. But the authors show that even when these derivatives are perfectly calculated mathematically, they might still miss important physical rules like direction of flow or energy conservation. This means AD might produce answers that look correct but don’t actually follow the physics of the problem. The authors suggest that future work should focus on ways to include the physical meaning into machine learning models, beyond just computing derivatives exactly.

What this means in practice

A position paper. It proposes an approach and reports no results.

Authors

Ameya D. Jagtap

Abstract

Automatic differentiation (AD) lets neural networks compute derivatives of governing equations to machine precision, and this precision has made it the computational backbone of physics-informed machine learning. Yet exactness in the mathematical sense is not the same as fidelity to the physics. Here I argue that a derivative can be numerically perfect and still be the wrong derivative for the problem at hand, because AD, by construction, has no notion of the physical structure a solution must obey. Convection and its associated directionality, diffusion, and dispersion are only the most visible instances of a much longer list that spans all branches of computational science and engineering, including conservation, thermodynamic consistency, symmetry, symplectic structure, positivity, monotonicity, and boundedness. Recognizing this broader gap reframes how the field should build the next generation of PDE-driven neural surrogates.
View PDFOpen arXiv