Spectral amplitude adjustment improves diffusion distillation speed and detail
Spectral Amplitude Purification in Distribution Matching for Diffusion Distillation
Computer Vision and Pattern Recognition
Summary
Diffusion distillation helps create detailed images quickly, but this process can get stuck focusing on broad, blurry parts and miss finer details. The authors found that certain low-frequency parts of the image data overshadow the finer features during training. They introduced a method called Spectral Amplitude Purification to balance these signals, allowing better focus on small details. This leads to faster training and sharper generated images.
What this means in practice
- •For ai developers: Create faster high-quality image generation tools with fewer sampling steps by improving diffusion distillation training.
- •For computer graphics teams: Produce more detailed visual textures in graphics applications by reducing low-frequency dominance during model training.
Authors
Zhenyu Zhou, Can Wang, Chun Chen, Zeyu Zheng, Defang Chen
Abstract
Distribution Matching Distillation (DMD) enables high-quality diffusion sampling in only a few steps, but its optimization dynamics remain dominated by coarse, low-frequency signals, delaying the recovery of fine-grained details. We identify a pronounced concentration of spectral amplitudes at low frequencies in the DMD directional error, where dominant low-frequency components overwhelm weaker mid- and high-frequency signals. To address this issue, we propose Spectral Amplitude Purification for Distribution Matching Distillation (SAP-DMD), a plug-and-play approach that adaptively modulates the amplitude spectrum of the DMD directional field. By suppressing the dominant tail of the amplitude spectrum, SAP-DMD reduces low-frequency dominance and promotes more effective recovery of fine structures and textures. Experiments on PixArt-$α$, SD3, and SD3.5 demonstrate that SAP-DMD accelerates training convergence and improves generation quality under both 2-step and 4-step sampling.