New method distills high quality images with less compute
Data-free Multistep Diffusion Distillation
Computer Vision and Pattern Recognition
Summary
Creating detailed images from text prompts with AI models can be slow and needs lots of data. The authors found that common ways to speed up this process face problems like unstable training and inconsistent results. They developed a new step-by-step training method called PMDD that uses less data and memory but still produces high-quality images quickly. This approach supports generating images with different amounts of computation while keeping the results reliable and clear.
What this means in practice
- •For machine learning engineers: Deploy faster text-to-image models that maintain image quality with less training data and compute resources.
- •For graphic design studios: Use AI image generation with more flexible speed-quality trade-offs for high-resolution artwork and concept design.
Authors
Ngoc Duy Tran, Kien Do, Duc Thanh Nguyen, Truyen Tran
Abstract
One-step diffusion distillation achieves strong performance on small datasets but remains insufficient for high-resolution or complex text-to-image settings. Extending to multistep distillation often requires training data, suffers from early-step degradation, inconsistent outputs across sampling budgets, and incurs high computational cost. We identify that these limitations stem from simultaneous training, where a single adapted network approximates multiple score functions of different student distributions at once, causing biased distribution matching and unstable optimization. Therefore, we propose Progressive Multistep Data-free Distillation (PMDD), which assigns a dedicated adapted network per student distribution while avoiding the memory overhead of maintaining multiple adapted networks. Combined with a stage-wise unforget loss to preserve earlier-step generations, PMDD reduces optimization bias in the student, provides faster convergence, and a single unified model that supports multiple inference budgets. Extensive experiments show that PMDD matches or outperforms data-free few-step baselines at every multistep budget, reaches FID 1.99 on ImageNet 64x64 in two steps, below its 512-NFE teacher (2.32), and achieves FID 7.76 on zero-shot COCO 512x512, while requiring 4-5x less training compute than data-free DMDv2 to support three inference budgets.