Image generation speed and quality improved by splitting work into two phases
Two Halves are More than One: Phase-wise Velocity Distillation for Fast and High-Quality Image Generation
Computer Vision and Pattern Recognition
Summary
Generating images using diffusion models usually takes many steps and lots of computer power. The authors found that trying to make a single fast model do everything at once often makes blurry pictures. They created a method that splits the job into two parts: a coarse phase for overall structure and a fine phase for details, each handled by a smaller specialized model. This approach makes image generation faster and produces better-quality images.
What this means in practice
- •For graphics software developers: Create faster image generation tools that deliver high-quality pictures within limited computing budgets by splitting generation into two specialized phases.
- •For cloud ai service providers: Reduce computational resources and memory usage for large-scale text-to-image generation services by using phase-wise velocity distillation methods.$Commercial implications: Enables more efficient, cost-effective text-to-image generation services, which can be sold as faster or cheaper AI-powered image generation offerings.
Authors
Zhen Guo, Rongyuan Wu, Qiaosi Yi, Chenxi Xie, Xinyu Wei, Lei Zhang
Abstract
Recent diffusion-based image generation backbones have grown substantially in scale, making the network inference cost increase rapidly. While diffusion distillation techniques can reduce the number of inference steps, high-quality image generation within a single full-backbone-forward compute budget remains challenging. Existing one-step methods typically allocate this budget to a single evaluation of a monolithic student. However, approximating the heterogeneous coarse-to-fine transport with a single monolithic mapping is difficult and often leads to over-smoothed outputs. To address this issue, we propose Phase-wise Velocity Distillation (PVD), which partitions the generation timeline into a coarse and a fine phase, and models the transition within each phase via the average velocity. A dedicated half-sized expert is assigned to each phase, decoupling structural composition from detail refinement while keeping the cumulative computation equivalent to one full-backbone forward pass. We show that the use of two half-sized phase-specific experts outperforms a single full-size monolithic student. On class-conditional image generation, PVD achieves an FID of 1.48 on ImageNet 256 x 256. On more complex text-to-image (T2I) tasks, PVD-distilled models (Stable Diffusion 3.5-Medium, FLUX.1-dev, Qwen-Image) produce results competitive with their multi-step teachers, significantly outperforming prior distillation methods. Moreover, across the evaluated T2I backbones, PVD reduces active parameters by 49.10-50.89% and peak VRAM by 45.76-48.36% compared to the corresponding teachers. Source code and distilled models are available at https://github.com/PolyU-VCLab/PVD.