Flow-Transformed Implicit Processes for Function-Space Variational Inference
2026-06-01 • Machine Learning
Machine Learning
AI summaryⓘ
The authors study a way to predict uncertain functions in Bayesian models where the usual mathematical descriptions are hard to get. They improve past work that combined a limited set of example functions using simple, bell-shaped distributions by allowing more complex shapes with a tool called normalizing flows. This lets their method better capture tricky uncertainties like asymmetries or multiple peaks in the predictions. They also use a special training method to balance covering all possibilities or focusing on main modes. Their experiments show this approach models complicated uncertainty better than older methods.
Bayesian inferenceimplicit processfunction-space modelingvariational inferencenormalizing flowposterior distributionGaussian distributionBlack-Box α objectivemultimodal distributionsasymmetric uncertainty
Authors
Luis A. Ortega, Andrés R. Masegosa, Thomas D. Nielsen
Abstract
Implicit-process priors define distributions over functions through flexible generative mechanisms, making them attractive for Bayesian function-space modelling. However, performing posterior inference with such priors is challenging because their induced function-space distributions are typically not available in closed form. One practical strategy is to approximate the prior using a finite collection of sampled functions, and then represent posterior functions as learned combinations of these samples. Existing approaches commonly place a Gaussian variational distribution over the combination weights. While tractable, this choice limits the shapes of posterior uncertainty that can be represented, especially when the true posterior is asymmetric, heavy-tailed, or multimodal. We propose Flow-Transformed Implicit Processes (FTIP), a variational inference method that makes this finite-dimensional function-space approximation more expressive. Instead of using a Gaussian distribution over the combination weights, FTIP uses a normalizing flow to define a richer variational distribution. This induces a flexible posterior distribution over functions while preserving tractable optimization. We train the model using a Black-Box α objective, allowing us to compare mass-covering and mode-seeking variational behaviour. Experiments show that FTIP captures asymmetric and multimodal posterior structure in function space that Gaussian coefficient approximations tend to smooth or collapse.