Guidance estimation improves steering of diffusion model sampling

WEIRDO: WEak resIdual Regularized DOob's h-transform diffusion alignment

Machine Learning

Summary

Diffusion generative models create data by gradually adding and removing noise. Sometimes, you want to guide these models to generate data that fits a slightly different target distribution. The authors study how to estimate the best guidance signal at inference time to steer the model properly, even when the exact target is complicated. They propose a new method that uses a penalty based on a mathematical condition called space-time harmonicity, which helps estimate this guidance more accurately and with good theoretical guarantees. Their approach works under minimal smoothness assumptions and is supported by experiments.

What this means in practice

  • For machine learning engineers: Improve sampling quality and control in diffusion models by better estimating guidance signals for targeted generation tasks.
  • For statistical modelers: Use the proposed penalized estimation to obtain accurate drift corrections when adjusting diffusion processes toward weighted target distributions.

Authors

Denis Suchkov

Abstract

We study the problem of estimating the guidance that steers the distribution learned by a diffusion generative model toward a tilted target $q_0 \propto w\,p_0$ at inference time. Relying on the stochastic optimal control approach, we observe that the exact drift correction is the gradient of the logarithm of Doob's $h$-function, and we study the problem of estimating it from a sample. In the present paper, we assume that the score of the pretrained model is available, that the tilting weight is bounded and positive, and that the reference distribution has a bounded support, no smoothness of the weight is required. Introducing a penalized least-squares risk in which the penalty is the residual of the space-time harmonicity equation satisfied by the $h$-function, measured in a dual Sobolev norm, we derive high-probability bounds on the squared error of the resulting guidance estimate. Since the penalty vanishes at the target, the estimator is free of regularization bias, and in favourable scenarios its rate of convergence is faster than the minimax rate of estimating first-order derivatives of a smooth regression function. Assuming that $w$ is bounded and positive with $\mathbb{E}_{p_0}[w^{-\mathrm{s}}] < \infty$ for some $\mathrm{s} \in (0,\infty]$, and that the reference data are compactly supported, we prove that the guidance is estimable in squared $L^2$ at rate $\varepsilon_n^{\mathrm{s}/(\mathrm{s}+4)}$, where $\varepsilon_n = n^{-2(β-1)/(2(β-1)+d)}.$ We also transfer the obtained bounds to the total variation distance between the marginals of the estimated and the exactly guided samplers, and illustrate the performance of the suggested approach with numerical experiments.