Papers for

bayesian data analysts

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Transport methods improve sampling from complex multimodal distributions

Transport along Local Annealing Paths: Wasserstein Geometry and Multimodal Sampling

Abstract: Variational approaches to learning transport maps for multimodal sampling can suffer from mode collapse. One way to mitigate this problem is to learn transport along a prescribed path of intermediate distributions defined by geometric annealing. We show that annealing can take a substantial detour in Wasserstein space: when mass moves between modes, the path can be much longer than the distance between its endpoints. We further link path geometry to transport learning, showing that long annealing paths can slow population optimization, while sufficiently regular paths permit rapid convergence. Motivated by these findings, we propose componentwise annealed transport (CAT) to reduce the need for mass transfer between modes. Starting from a mode-informed mixture reference, CAT constructs an exact mixture decomposition of the target and learns separate transport maps along componentwise annealing paths. We estimate the component weights separately and show that these estimates remain consistent even when the learned flows are inexact. We also develop adaptive annealing schedules guided by transport effort. Experiments on multimodal synthetic distributions and Bayesian inference problems demonstrate improvements in transport learning and sampling efficiency.

Tue 3 NovMachine Learning
The gist
Sampling from complicated distributions that have multiple peaks can be hard because methods often get stuck on some peaks and miss others. The authors show that moving samples along a path between simple and complex distributions can be inefficient, especially if mass moves between distant modes. They propose a new method called componentwise annealed transport (CAT) that breaks down the problem by targeting each mode separately, which helps with learning the transport maps and speeds up sampling. Their experiments show that this approach improves sampling from challenging multimodal cases.
Open → 2611.05605v1

Denoising method improves sampling efficiency in variational inference

Denoising Implicit Variational Inference

Abstract: Semi-implicit variational inference (SIVI) expands the representational capacity of variational families, but optimizing the Evidence Lower Bound (ELBO) remains challenging because the marginal score of the semi-implicit distribution is generally intractable. Unbiased Implicit Variational Inference (UIVI) and related methods address this difficulty through reparameterized ELBO gradients, but existing approaches estimate the required score indirectly through reverse-conditional sampling or approximation, which can be computationally costly and unstable. We propose Denoising Implicit Variational Inference (DIVI), which learns the marginal score directly via denoising score matching. The learned score is used as a plug-in estimator in the pathwise ELBO gradient, replacing inner reverse-conditional sampling with a score-network evaluation. We analyze DIVI as inexact stochastic gradient ascent and show, under stated assumptions, that the averaged stationarity measure is controlled by the usual optimization term and the average score-matching error. Empirically, we evaluate DIVI on both synthetic distributions and a variety of real-data Bayesian inference tasks. The results show that DIVI improves over the current UIVI baselines while reducing the cost of ELBO-gradient estimation.

Sat 17 OctMachine Learning
The gist
Variational inference is a technique to estimate complex probability distributions but can be slow and unstable when trying to improve accuracy. The authors propose a new method called Denoising Implicit Variational Inference (DIVI) that learns an important mathematical component directly, making the process more reliable and less costly. They tested DIVI on simulated and real-world data, showing it works better than previous methods. This method helps create better models for understanding uncertainty in data.
Open → 2610.24509v1

Symbolic formulas speed up neural likelihood inference by more than 100 times

Recovery-Directed Symbolic Distillation of Neural Likelihoods

Abstract: Amortized neural likelihoods enable computationally expensive inference for models with analytically intractable or unspecified likelihoods, but their black-box nature limits interpretability. We introduce a symbolic distillation pipeline that converts trained neural likelihoods into explicit, interpretable expressions optimized for efficient parameter estimation. Our approach uses a recovery-directed objective to guide symbolic regression toward expressions that preserve parameter-recovery accuracy rather than merely approximating the likelihood function. Candidate expressions are evaluated on held-out datasets and selected using a criterion that jointly accounts for expression complexity, parameter-recovery performance, and distributional distance from the learned likelihood. We evaluate the pipeline on the diffusion decision model, a classical cognitive model, whose analytically tractable likelihood provides ground truth for controlled evaluation. The proposed recovery-directed objective improves parameter recovery over standard symbolic-regression objectives. The resulting symbolic likelihoods enable over 100 times faster parameter evaluation than both neural likelihoods and, when available, the exact likelihood, while maintaining a manageable loss in precision. We further demonstrate these computational benefits in Bayesian hierarchical inference on empirical data. Our pipeline provides a lightweight interface for integrating symbolic distillation with existing neural-likelihood estimation methods and can be adapted to a range of simulation-based inference settings.

Sat 26 SeptMachine Learning
The gist
Some models need complicated math to figure out how likely certain outcomes are, but this math can be slow and hard to understand. The authors show how to turn these complicated neural network calculations into simple, clear formulas that are much faster to use. They make sure these formulas still help find model parameters accurately. In tests with a known model, their method speeds up calculations a lot while keeping good accuracy.
Open → 2609.32409v1