Papers for

empirical bayes practitioners

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

NPMLE achieves optimal rates for estimating Gaussian mixtures

Minimax Optimality of the NPMLE for Gaussian Mixtures in Hellinger Distance

Abstract: We analyze the problem of estimating univariate Gaussian mixture densities using the nonparametric maximum likelihood estimator (NPMLE) based on $n$ independent and identically distributed samples. We show that the unconstrained NPMLE achieves the minimax optimal rate $\log(n)/n$ in squared Hellinger distance, when the true mixing measure is sub-Gaussian. In addition, we show that when the true mixing measure has compact support, the NMPLE constrained to a compact interval also achieves the minimax optimal rate, which is slightly faster and of order $\log(n)/(n \log \log n)$. Our proof consists of sharply characterizing the local Hellinger bracketing entropy of the class of finite Gaussian mixtures. One of our key technical innovations is to sharply bound the $L^2$ bracketing entropy of balls of bounded-degree polynomials, which may be of independent interest. As an application, we establish the regret-optimality of $g$-modeling procedures based on the NPMLE in the Gaussian Empirical Bayes problem with bounded priors.

Wed 21 OctInformation TheoryMachine Learning

The gist

Estimating a mix of bell curves (Gaussian mixtures) from data can be tricky. This paper shows that a certain estimation method, the nonparametric maximum likelihood estimator (NPMLE), performs as well as theoretically possible in measuring how close the estimate is to the true distribution. The authors prove this works well even when the data comes from mixtures with bounded or sub-Gaussian spreads. They use advanced mathematical tools to understand the complexity of these mixture models and also show how this improves beyond statistics, helping in areas like Empirical Bayes techniques.

Open → 2610.30109v1