Quenched ensemble sampling improves modeling across physical transitions
Quenched Ensemble Sampling
Machine Learning
Summary
Sampling complex physical systems near transitions where properties change suddenly is very challenging because many methods get stuck. The authors introduce Quenched Ensemble Sampling, a technique that gently guides sampling through these tricky regions using repelling forces instead of strict limits. This method works better in high-dimensional problems and helps estimate important quantities like probabilities and partition functions where older methods fail. They show it works well on models simulating phase changes and in Bayesian neural networks for comparing architectures.
What this means in practice
- •For machine learning engineers: Compare neural network models by estimating marginal likelihoods even when model energy landscapes include abrupt transitions.
- •For computational physicists: Calculate partition functions and sample states in lattice field theories that undergo first-order phase transitions more reliably.
Authors
David Yallup
Abstract
Some of the sharpest challenges in sampling from the energy functions of physical systems arise at phase transitions, where the density of states changes abruptly and many sampling algorithms stall. Nested sampling is a particle method that traverses the density of states under a hard energy constraint and is known to be robust to such transitions, but its application in high dimension is limited by the difficulty of sampling under that constraint. In this work we introduce Quenched Ensemble Sampling, which generalises the hard constraint to a family of repulsive potentials at the energy boundary. This preserves the quenched path of monotonically decreasing energy while making the constrained target amenable to scalable gradient-based kernels. We demonstrate on synthetic models of phase transitions that our method estimates the marginal likelihood and draws posterior samples across a first-order transition where popular alternatives such as tempering fail. We apply the procedure to marginal likelihood estimation in Bayesian neural networks, enabling model comparison between network architectures. Finally, in a high-dimensional continuous lattice field theory, we show that this method traverses a first-order transition and estimates the partition function.