Momentum SVGD-EM for Accelerated Maximum Marginal Likelihood Estimation
2026-03-09 • Machine Learning
Machine Learning
AI summaryⓘ
The authors look at a way to improve how computers estimate certain model settings through a method called Maximum Marginal Likelihood Estimation (MMLE). They view a popular algorithm, Expectation-Maximisation (EM), as a step-by-step approach that tweaks both the model and probability guesses. Building on this, they improve a recent related technique that uses many particles to find better estimates faster. Their new method, called Momentum SVGD-EM, adds a speed-up trick known as Nesterov acceleration and works well across simple and complex problems.
Maximum Marginal Likelihood EstimationExpectation-Maximisation (EM) algorithmfree energy functionalcoordinate descentStein Variational Gradient Descent (SVGD)Nesterov accelerationparticle methodsprobability measuresparameter estimationconvergence acceleration
Authors
Adam Rozzio, Rafael Athanasiades, O. Deniz Akyildiz
Abstract
Maximum marginal likelihood estimation (MMLE) can be formulated as the optimization of a free energy functional. From this viewpoint, the Expectation-Maximisation (EM) algorithm admits a natural interpretation as a coordinate descent method over the joint space of model parameters and probability measures. Recently, a significant body of work has adopted this perspective, leading to interacting particle algorithms for MMLE. In this paper, we propose an accelerated version of one such procedure, based on Stein variational gradient descent (SVGD), by introducing Nesterov acceleration in both the parameter updates and in the space of probability measures. The resulting method, termed Momentum SVGD-EM, consistently accelerates convergence in terms of required iterations across various tasks of increasing difficulty, demonstrating effectiveness in both low- and high-dimensional settings.