New method improves function prediction with complex prior models
Generalized Matheron Variational Implicit Processes
Machine Learning
Summary
Predicting unknown functions is tricky when relying on complex models that don't directly give their probabilities. The authors introduce GMVIP, a new way to make predictions using such complex models by improving how these models are corrected and adjusted. This method keeps the original patterns and variability of the prior models while making inference more practical. It works well on tasks like regression, classification, and forecasting with different kinds of data sources.
What this means in practice
- •For machine learning engineers: Improve probabilistic predictions in models defined by complex simulators or neural networks without explicit densities.
- •For forecasting teams: Generate better uncertainty estimates for time series forecasting using flexible empirical trajectory priors.
Authors
Luis A. Ortega, Andrés R. Masegosa, Thomas D. Nielsen
Abstract
Implicit-process priors specify distributions over functions through sample-forward mechanisms such as Bayesian neural networks and stochastic simulators, but their function-space densities are typically unavailable. We introduce Generalized Matheron Variational Implicit Processes (GMVIP), a pathwise variational family for posterior inference with such priors. For Gaussian-process priors, GMVIP recovers the standard inducing-variable variational GP construction; for general implicit priors, its empirical covariance construction preserves the prior mean and covariance in the population limit. GMVIP constructs posterior samples by drawing a function from the prior and applying a correction anchored at a set of inducing inputs. The effect of this correction away from the inducing inputs is determined directly from prior samples, allowing the posterior to retain the structure and variability of the original implicit process. The (surrogate) prior and variational posterior use the same pathwise construction and differ only in the distribution of whitened inducing coefficients, yielding a tractable coefficient-space Kullback-Leibler divergence. Experiments on regression, classification, and forecasting with simulator-defined and retrieval-conditioned empirical trajectory priors show that GMVIP is broadly competitive with existing methods.