Neural operators learn best with fractional derivatives for faster pde solutions
Fractional is Better: Learnable Derivative Orders in Neural Operator Learning
Machine Learning
Summary
Solving complex equations called partial differential equations (PDEs) is usually slow. This paper shows that teaching neural networks to use not just raw data points but also fractional derivatives—like partial rates of change—helps the networks learn better and faster. Interestingly, the best amount of derivative information is often less than what the original equation suggests. The authors built a flexible method that adds these learnable derivative features to many neural networks, improving accuracy in tests.
What this means in practice
- •For engineering simulation teams: Speed up and improve the accuracy of PDE-based simulations by integrating learnable fractional derivative features into surrogate models.
- •For climate model developers: Enhance prediction accuracy in climate simulations by using neural operators augmented with fractional derivatives that adaptively capture physical dynamics.
Authors
Fares B. Mehouachi, Saif Eddin Jabari
Abstract
Neural operators learn mappings between function spaces, enabling fast surrogate solutions to partial differential equations. Despite remarkable architectural diversity, these methods often share a common input representation: raw coordinate-value pairs $(x, u(x))$. We ask whether inputs aligned with PDE differential structure can improve learning. Through Picard iteration on mild solutions, we show that derivatives of the input appear explicitly in the solution operator, suggesting that providing derivative features should reduce the network's implicit differentiation burden. We prove this intuition: providing derivative features improves approximation bounds from $O(W^{-(s-m)/d})$ to $O(W^{-s/d})$, where $W$ is parameter count, $s$ is input regularity, $m$ is the PDE order, and $d$ is spatial dimension. Our central finding, however, is a surprise: the optimal derivative order $β^*$ is strictly less than the PDE order $m$. This gap arises from a bias-variance tradeoff in spectral space that we characterize in closed form. Learning $β$ from data achieves automatic spectral regularization. We introduce $\partial$-NO (del-NO), for derivative-augmented neural operators, an architecture-agnostic augmentation that provides learnable fractional derivative features to any neural operator backbone. Across benchmark problems and architectures, $\partial$-NO consistently improves prediction accuracy, with learned orders that reflect a representation of known physics modulated by noise and finite-sample constraints.