Generalized Splines and Gaussian Processes
Machine Learning
Summary
The authors show that a well-known way to estimate unknowns in problems with Gaussian variables can be extended from simple finite-dimensional cases to much more complex infinite-dimensional ones. They use advanced mathematical tools like generalized splines and Gaussian processes in these broader settings. Their approach unifies many existing methods and connects classic ideas about functions to modern concepts like distributions. This helps in solving complex inverse problems using Bayesian methods in very general scenarios.
Authors
Michael Unser
Abstract
For finite-dimensional linear inverse problems where the variables are Gaussian, it is well-known that the minimum-mean-square error estimator takes the form of a regularized least-squares data fit. In this chapter, we show that this equivalence extends to a much broader infinite-dimensional setting where generalized splines take the role of linear regressors and generalized Gaussian processes on a nuclear space $S$ are the counterpart of Gaussian random vectors. The scope of this extension is of the same nature as the switch from the classic notion of function to that of a distribution, also known as a "generalized function." Our formalism involves a whitening/regularization operator $L: S\to S'$ whose continuous extension induces a native Hilbert space $H\subset S'$ that plays a central role in our characterization. The presentation is self-contained for the most part and remarkably general and powerful. It allows for the recovery of all known instances of such equivalences; in particular, the methods involving innovations and reproducing-kernel Hilbert spaces developed by Kailath and his students, and the mathematical correspondence between fractional splines and Mandelbrot's fractional Brownian motion (fractals), with the former being the optimal estimators of the latter. It also covers general Bayesian methods for the resolution of infinite-dimensional inverse problems.