AI summaryⓘ
The authors develop new mathematical methods to understand how well deep learning models with multiple outputs can generalize to unseen data. They use advanced operator theory by viewing neural network layers as special transformations acting on functions within certain structured spaces called vector-valued reproducing kernel Hilbert spaces. Their work provides bounds that measure complexity, separating contributions from the outputs and the network layers, and applies to different types of function spaces with distinct technical properties. They also explore learning shared components across tasks and demonstrate theoretical results connecting to practical algorithms, supported by experiments on synthetic data and handwritten digit recognition. Overall, their results give precise formulas to analyze and bound the capacity of complex multi-output networks, without making broad dominance claims between different function spaces.
Koopman operatorsreproducing kernel Hilbert spacesRademacher complexitySobolev spacesBrownian motionCameron–Martin spacemulti-output neural networksoperator normsfinite-rank representer theoremtarget-transfer bound
Authors
Mahdi Mohammadigohari, Thomas Borsani, Giuseppe Di Fatta
Abstract
We develop operator-theoretic generalization bounds for deep multi-output function classes by representing network layers as Koopman composition operators on vector-valued reproducing kernel Hilbert spaces. In vector-valued Sobolev RKHSs, we derive Rademacher complexity bounds for invertible and width-expanding injective architectures. The estimates separate the output-coupling contribution, represented by the trace of the task matrix, from the layerwise operator norms, Sobolev symbol ratios, determinant factors, and restriction constants generated by the linear maps. We then analyze a distinct one-dimensional Brownian/Cameron--Martin regime. Using the exact anchored derivative-norm characterization of the vector-valued Brownian RKHS, we obtain layerwise bounds for domain-preserving scalar linear maps and anchored diffeomorphic activations; the corresponding factors scale as $|W_l|^{1/2}$ and $\|σ_l'\|_\infty^{1/2}$, respectively, and do not involve Sobolev smoothness exponents. Because the Sobolev and Brownian results concern different hypothesis spaces, neither is asserted to dominate the other uniformly. We additionally formulate shared operator learning across tasks, prove a finite-rank representer theorem, derive the exact finite-dimensional problem for squared loss, and establish a target-transfer bound when the learned operator is obtained independently of the target sample. Synthetic and MNIST studies examine stabilized Sobolev-inspired and Brownian-inspired complexity proxies; these empirical proxies are not evaluations of the proved bounds for rank-deficient architectures.