Convex equilibrium networks give stable solutions for nonlinear equations
Convex Equilibrium Potentials for Monge--Ampère: Curvature Bounds and A Posteriori Evaluation
Machine Learning
Summary
Solving complex nonlinear equations like the Monge--Ampère equation often requires deep neural networks, but deeper networks can be unstable or hard to evaluate precisely. The authors study a special kind of neural network called a convex equilibrium network, which finds stable fixed points without relying on fixed depth. They prove that the network’s curvature behaves well under certain contraction constraints and provide methods to rigorously check how close approximate solutions are to true solutions. This approach was tested on example problems, successfully certifying accuracy and improving evaluation efficiency.
What this means in practice
- •For scientific computing teams: Provide rigorous error bounds and certified solution accuracy for neural network solvers of nonlinear PDEs like Monge--Ampère.
- •For numerical simulation developers: Enable adaptive evaluation stopping criteria that guarantee solution quality in equilibrium neural network PDE solvers while improving runtime efficiency.
Authors
Edison Vázquez, Yingdong Lu, Tomasz Nowicki
Abstract
Equilibrium neural networks define their hidden state as the fixed point of a contraction, so their effective depth is set at evaluation time rather than by the architecture. Using them for fully nonlinear PDEs raises two issues that explicit networks avoid: second derivatives may grow with the effective depth, and the potential is only ever evaluated approximately. We study nonnegative softplus equilibrium networks with contraction factor $q<1$, a convex class suited to the Monge--Ampère equation. A positivity cancellation in the resolvent shows that the Hessian grows at most like $(1-q)^{-2}$, rather than the $(1-q)^{-3}$ of a direct estimate, and an explicit one-neuron family shows that this exponent is sharp; third derivatives satisfy a sharp $(1-q)^{-3}$ bound. Residual-based a posteriori estimates then control the state, its input derivatives and, in two dimensions, the Hessian determinant at an approximate equilibrium. Combined with interval arithmetic on complete covers of the domain, they yield rigorous uniform bounds on the Monge--Ampère residual and, through a comparison argument, on the solution error of a fixed trained network. On two manufactured Dirichlet problems on the unit disk, all six trained models are certified with residual below $0.25$ and boundary error below $0.01$. Adaptive stopping halves float64 evaluation time at best, whereas certified adaptive evaluation remains costlier than fixed depth.