Likelihood approximation using chained normalizing flows for simulation inference

A perspective note on likelihood approximation and inference for complex simulation models using a chain of aggregated normalizing flows

Machine Learning

Summary

Complex simulations often generate data that is hard to analyze because their likelihood functions are difficult to compute. The authors propose using a series of linked mathematical transformations called normalizing flows to approximate these likelihoods more efficiently. By sequentially estimating parameters of these flows, their approach allows for faster and more flexible exploration of possible model settings. This method also supports statistical tasks like hypothesis testing and measuring uncertainty in predictions.

What this means in practice

  • For computational scientists: Generate reliable surrogate models for complex simulations to speed up data analysis and hypothesis tests.
  • For statistical modelers: Use chained normalizing flows for efficient parameter exploration and uncertainty quantification in high-dimensional inference tasks.

Authors

Getachew K Befekadu

Abstract

We present a new perspective on the problem of likelihood approximation within the framework of simulation-based inference that promotes scalable and controllable simulation routines for large-scale data analysis, allows efficient parameter space exploration or smooth interpolation in high-dimensions and, thus, supports valid statistical treatments of hypothesis testings as well as uncertainty quantification. In particular, we consider a chain of $n$-aggregated normalizing flows for likelihood approximation scheme, where a set of upfront replicated observation datasets from the forward complex simulation model pass through the first set of bijective transformations, and then subsequently pass to the other sets of bijective transformations. Here, we assume that, for any $k \in \{1,\,2, \ldots, n\}$, the parameters corresponding to the first $k$ sets of bijective transformations are estimated sequentially, in some sense of optimality, for constructing flexible probability distributions, regardless of the remaining $(n-k)$ sets of bijective transformations. Moreover, our objects of interest are to highlight two complementary mathematical arguments that leverage an informatics-theoretic formalization, based-on empirical likelihood estimators under moment restrictions, and a sequential decision-making paradigm, with mixing distributions, for updating and aggregating the estimated parameters of the overall normalizing flows. As a by-product, the framework provides a reliable surrogate model, conditioned on the model parameters defining the forward computational simulation, that allows samples generation, with statistical powers, and facilitates computationally tractable scheme in the Bayesian paradigm for inference, hypothesis testings and uncertainty quantification.