Posterior matching distillation speeds image generation with better diversity
Posterior Matching Distillation
Computer Vision and Pattern Recognition
Summary
Generating images step-by-step with AI models can be slow and sometimes misses less common but important details. The authors propose a new way called Posterior Matching Distillation (PMD) to teach a smaller, faster model to copy the full range of possible images the slower model creates. PMD captures the whole range of possible outputs without extra helper networks, allowing fast generation at different speeds without retraining. Tests show PMD improves image quality and keeps good variety even with very few steps.
What this means in practice
- •For image generation developers: Create fast image generation tools that keep diverse, high-quality outputs using fewer inference steps than prior methods.
- •For video content creators: Produce quick video frame generation maintaining variety and quality by integrating PMD into existing diffusion-based video pipelines.
Authors
Hidir Yesiltepe, Julius Berner, Arash Vahdat, Pinar Yanardag
Abstract
Diffusion distillation accelerates image and video generation by compressing a pretrained teacher's iterative sampling process into a few network evaluations. Distribution-based methods achieve this by matching the student's output distribution to the teacher's. This matching is often driven by reverse-KL objectives, whose mode-seeking bias favors dominant teacher modes at the expense of diversity across samples. We introduce Posterior Matching Distillation (PMD), a new member of the distribution-based distillation family that combines mode-covering posterior sampling with any-step generation. At zero population loss, PMD recovers the teacher's full posterior of clean images at each trained noise level. Its stochastic student supports any-step sampling budgets within the trained timestep grid without further training. To learn these posteriors, PMD matches exponentially reweighted averages of student samples conditioned on a fixed noisy state to corresponding teacher posterior means. Under Gaussian corruption, exponentially tilted posterior mean is a single teacher evaluation at a shifted input. We prove that, matching tilted means over an open neighborhood uniquely identifies the full posterior. PMD thus supervises the entire conditional distribution through a family of mean constraints, without any auxiliary student-score network. Experiments on SDXL and FLUX.2 [klein] 4B demonstrate that PMD achieves the best FID among 4-step methods on both models, while maintaining competitive or leading performance across prompt-alignment and perceptual-quality metrics.