Normalizing flows find key statistics even with nuisance parameters
Likelihood-free inference with nuisance parameters through normalizing flows
Machine Learning
Summary
Many scientific problems have important variables mixed with nuisance factors that are not of direct interest. The authors show a way to use neural networks called normalizing flows to separate out key statistics that behave reliably despite those nuisance factors. This method only needs the ability to generate sample data and can handle prior knowledge like shifts or scaling in data. The technique matches or beats classical tests in accuracy and speed on small to moderate data samples.
What this means in practice
- •For data scientists: Obtain reliable test statistics for inference from complex simulations without explicit likelihood calculations.
- •For medical statisticians: Improve hypothesis testing accuracy on small clinical trial datasets where nuisance variation is present.
Authors
Phil Assheton
Abstract
We present a simple decomposition of a neural-network-based normalizing flow that naturally uncovers a pivotal statistic (or something close) in the presence of nuisance parameters, based only on a sample generator from the distribution of interest. We show that the statistic is near-pivotal in the sense of minimum average KL-divergence of its $p$-values versus uniform and we argue that it can be expected to have good power when the dimension of the statistic equals the dimension of the parameter. It is able to incorporate prior knowledge about group invariances such as translation and scale. It can discover the one-sample $t$-test almost exactly, outperforms the Welch test in terms of worst-case size over a constrained variance-ratio range and achieves good calibration on partial biserial correlations, while showing higher power (and being much faster) on small-to-moderate samples than profile likelihood-ratio techniques.