Consistent distribution matching speeds up diffusion model generation
Consistent Distribution Matching for Data-Free Diffusion Distillation
Machine LearningArtificial IntelligenceComputer Vision and Pattern Recognition
Summary
Diffusion and flow models generate data like images but usually take a long time to create each one. The authors propose a new method that speeds up this process by teaching a simpler model to mimic a complex one without needing extra training data or simulations. They combine sample creation and learning into one step, making the process more efficient. Their technique works well on high-resolution images and improves on past methods.
What this means in practice
- •For machine learning engineers: Develop faster image generation pipelines by distilling slow diffusion models into efficient one-step models without extra training data.
- •For graphics software developers: Integrate accelerated generative models in software to enable quicker rendering of high-quality images.
Authors
Yuxiang Fu, Qi Yan, Zike Wu, Yongxing Zhang, Purang Abolmaesumi, Lele Wang, Renjie Liao
Abstract
Flow and diffusion models suffer from slow inference due to computationally expensive numerical integration. Distillation provides a promising way for a student model to learn from a teacher's dynamics, enabling one-step or few-step generation. However, existing methods often depend on curated distillation datasets, costly teacher rollouts, or auxiliary proxy networks, which complicate model training and scaling. In this work, we propose Consistent Distribution Matching, a simulation-free and data-free distillation method for accelerating diffusion and flow models while preserving strong generative capacity. Our key insight is to unify sample generation and score estimation with one student network. Thus, our framework uses only two models, a frozen teacher and a trainable student, and optimizes one objective. We prove that minimizing our objective indicates Wasserstein convergence of the student flow-map pushforwards to the teacher marginals. On ImageNet 256$\times$256, our method attains an FID of 2.04 with a single function evaluation (1-NFE) and a 4-NFE FID of 1.37 within 40 epochs of training, surpassing the state-of-the-art distillation baselines without data. Our code code and model are available at https://consistentdmd.github.io/.