Automatic tuning of guidance weights improves diffusion posterior sampling
Getting Your Guidance Weights Right in diffusion and flow-matching posterior sampling
Machine Learning
Summary
Solving inverse problems using pretrained diffusion or flow models requires balancing between prior knowledge and new measurements, which depends on guidance weights. The authors found a simple way to automatically choose these weights by turning the problem into an offline math calculation, without needing to retrain the models. This makes the method work better and faster, reducing the number of steps needed for good results. Their method applies broadly to different sampling approaches and improves reconstruction quality.
What this means in practice
- •For medical imaging developers: Improve inverse image reconstruction quality and speed using pretrained generative models with optimized guidance weights tuned offline.
- •For remote sensing analysts: Enhance reconstruction from noisy sensor measurements by automatically tuning guidance weights in diffusion-based posterior samplers for better accuracy and efficiency.
Authors
Liam Moroy, Jean-François Giovannelli, Yoann Altmann, Steve McLaughlin, Frédéric Champagnat, Guillaume Bourmaud
Abstract
Training-free posterior sampling methods, also known as Plug-and-Play methods, leverage pretrained unconditional diffusion or flow-matching models to solve inverse problems. Most existing approaches rely on guidance weights to balance, at each time step, prior information from the unconditional score or velocity network with measurement consistency, yet the tuning of these weights is often not discussed and is largely left to heuristics. We introduce a simple and principled offline strategy for automatically tuning these guidance weights. Our key observation is that, at each time step, the conditional denoising score-matching objective for diffusion models, or the conditional flow-matching objective for flow-matching models, is a least-squares objective. Therefore, when the conditional prediction is expressed as a weighted sum of the unconditional network output and a measurement-guidance term, optimizing over these weights reduces to a two-dimensional linear least-squares problem. The resulting time-dependent guidance weights can be optimized offline for a given measurement operator, noise level and sampler at the cost of a single minibatch of sampling trajectories, without retraining or fine-tuning the pretrained generative model. Instantiated with the standard Tweedie-based measurement-consistency term, our approach improves posterior sampling and achieves state-of-the-art reconstruction performance across diffusion- and flow-matching-based methods. Moreover, the optimized guidance weights enable diffusion samplers to reduce the number of sampling steps from 1000 to 50 with no significant degradation in reconstruction quality. Code will be made available.