Interacting particle guidance improves adaptive sampling for generative models
Interacting particle guidance for sampling reward-tilted generative priors
Machine Learning
Summary
Sometimes computers need to create new images, sounds, or structures that meet specific goals without being retrained from scratch. The authors describe a way to guide this generation process with many interacting 'particles' that share information instead of working independently and reweighting themselves. This makes the sampling from complex models more stable and efficient, so the generated results better match the desired properties. They tested their approach on tasks like filling missing parts of images and predicting protein structures.
What this means in practice
- •For machine learning engineers: Generate customized synthetic data or images by guiding pretrained generative models toward specific outcomes without retraining.
- •For bioinformatics teams: Improve protein structure prediction by sampling more relevant configurations efficiently using adaptive particle guidance.
Authors
Adhithyan Kalaivanan, Zheng Zhao, Jens Sjölund, Fredrik Lindsten
Abstract
Inference-time steering adapts pretrained diffusion and flow-based models to new tasks, e.g., to generate samples from a conditional distribution or samples with desired properties, without retraining. This can be formalized as sampling from a reward-tilted generative prior. As exact sampling from this distribution is intractable, guidance-based methods rely on approximations producing biased samples, and sequential Monte Carlo (SMC) methods correct for this bias using importance weights. However, while exact in the large particle limit, SMC suffers from weight degeneracy and particle collapse in practice. We propose interacting particle guidance (IPG), which replaces reweighting with transport. The particles interact through an additional drift, derived from the Feynman--Kac PDE to cancel the reweighting term, and remain unweighted. Choosing the drift in a reproducing kernel Hilbert space yields a closed-form solution that is cheap to compute, with negligible overhead compared to SMC. We demonstrate the method on Gaussian mixtures with known posteriors, and on high-dimensional image inpainting and protein structure inference tasks.