A Statistical Formulation Gap for Nonlinear Multiscale Physics-Informed Learning
Machine Learning
Summary
The authors study how well neural networks can solve special physics problems with tiny complex details that vary on very small scales (controlled by ε). They find that some methods that directly measure errors have a built-in difficulty that gets worse when ε is small and the number of samples N is limited. In contrast, using a related approach based on minimizing a variational energy avoids this particular sampling difficulty, although it does not fully solve the challenge of approximating the tiny details. Their results include theoretical bounds and numerical experiments showing these effects in different dimensions.
Authors
Ronald Katende
Abstract
We prove a finite-sample formulation gap for physics-informed learning of nonlinear multiscale elliptic equations. For a uniformly monotone divergence-form class with coefficients oscillating at scale $ε$, we derive a finite-width, finite-sample, and finite-iteration error bound for a boundary-compatible variational neural solver. Its stability, sampling, and optimization constants are independent of $ε$, while all unresolved scale dependence is isolated in the best-approximation error. We then construct a minimal obstruction witness for a one-dimensional periodic diffusion equation with cubic reaction. Already on the one-parameter family $v_c(x)=cx(1-x)$, the empirical Rademacher complexity of the strong residual is bounded below by a constant multiple of $(ε\sqrt{N})^{-1}$, while that of the squared strong-residual loss is bounded below by a constant multiple of $(ε^2\sqrt{N})^{-1}$. These are optimizer-independent properties of the sampled residual and loss classes, rather than neural-tangent-kernel conditioning statements. The corresponding variational energy complexity is bounded above by a constant multiple of $N^{-1/2}$ uniformly in $ε$. A tensor-product construction shows that the same obstruction rates persist in every spatial dimension. Numerical evaluation gives fitted exponents $0.9971$, $1.9860$, and $-0.0028$ for the strong residual, squared strong loss, and variational energy, respectively. Thus, differentiating the microscopic coefficient creates finite-sample statistical ill-conditioning. The variational formulation removes this statistical penalty but does not remove the separate multiscale approximation problem.