Operator learning methods remain accurate as data resolution improves
Resolution-Independent Analysis of Encoder--Decoder Operator Learning via Limiting Kernels
Machine Learning
Summary
Learning how to approximate complex processes often deals with functions rather than fixed-size data. The authors studied how encoding and decoding functions from data of varying resolution affect learning accuracy. They found that as the resolution grows finer, the learning behaves consistently thanks to a limiting mathematical structure called the kernel. This guarantees that their methods work well even when data is represented in increasingly detailed ways. They also showed how their results apply specifically to certain neural network designs and common function encoding schemes.
What this means in practice
- •For machine learning engineers: Design and analyze encoder-decoder models that maintain accuracy as data resolution increases by understanding encoding-related error bounds.
- •For computational physicists: Use encoder-decoder operator learning methods with guarantees on error behavior when simulating physical systems at multiple resolutions.
A theory result. No direct application yet.
Authors
Lei Shi, Jia-Qi Yang, Ding-Xuan Zhou
Abstract
Operator learning is formulated on function spaces, but training data are typically available only through finite-dimensional representations. In encoder--decoder architectures, a matrix-valued kernel on the encoded space induces an operator-valued kernel on the original function spaces, and the corresponding reproducing kernel Hilbert spaces are isometrically isomorphic. As the input and output resolutions increase, the induced kernels converge to a limiting kernel, in the sense of operator-norm convergence of their associated integral operators, allowing regularity assumptions to be stated independently of the encoding resolution. For regularized stochastic gradient descent, we establish upper bounds for decreasing and fixed step sizes, separating the encoding and regularization terms from optimization terms of order \(t^{-θ}\) and \(T^{-θ'}\), respectively, for any \(θ,θ'\in(0,1)\). We further prove lower bounds showing that these encoding-induced terms are generally unavoidable. The analysis is further extended to encoder--decoder neural networks through the limiting neural tangent kernel (NTK), yielding error bounds with an additional finite-width term and polynomial parameter and sample complexity guarantees when the encoding errors decay algebraically. The framework covers matrix-valued kernels constructed from radial and dot product kernels, NTKs arising from wide encoder--decoder neural networks, and encoder--decoder pairs based on Fourier, Legendre polynomial, wavelet, PCA, or pointwise sampling representations.