Video diffusion models improved with projected distribution matching
PDMD: Projected Distribution Matching Distillation for Video Diffusion Models
Computer Vision and Pattern RecognitionMachine Learning
Summary
Video diffusion models create videos by gradually refining noisy images, but this process can be slow and sometimes produces poor quality or unnatural effects. The authors identify that errors in a key part of the training process called the critic cause these problems to build up over time. They introduce a new method, PDMD, that filters out these errors to improve stability and video quality without adding complexity or extra steps. Their method works better than previous approaches in tests, making shorter video generation processes produce clearer and more natural results.
What this means in practice
- •For video production teams: Generate higher-quality video content faster by reducing processing steps in diffusion-based video models.
- •For audio-visual content developers: Improve joint video and audio generation quality in multimedia projects using a stable and efficient distillation technique.
Authors
Zimo Wang, Junkun Yuan, Angtian Wang, Haotian Yang, Canyu Zhang, Siyuan Yuan, Xingchang Huang, Bo Liu, Yizhi Wang, Yiding Yang, Chongyang Ma, Gordon Guocheng Qian
Abstract
Modern video diffusion models require tens of denoising evaluations over long spatiotemporal token sequences. Distribution Matching Distillation (DMD) reduces the number of function evaluations (NFE) to just a few. However, DMD samples can degrade during training, exhibiting progressive oversaturation and artifacts. We trace this instability to critic errors, which enter successive student updates and accumulate over time. We introduce Projected Distribution Matching Distillation (PDMD) to filter critic errors. PDMD projects out the component of the DMD update parallel to the student-critic endpoint residual. At a fixed noisy query, we prove that this residual is an unbiased estimate of the critic's endpoint error. Under high-dimensional assumptions, this projection removes a constant fraction of critic error while discarding only a vanishing fraction of ideal DMD signal. Empirically, the projection stabilizes training and improves sample quality where DMD degrades and develops unnatural textures. PDMD requires only a one-line code change to DMD, with no extra loss, network, data, model pass, or multi-stage training. With Wan2.1, PDMD achieves a VBench total score of 83.73 at 4 NFE, surpassing matched DMD by 1.03 points. On MiniMax-H3 joint video-audio generation, PDMD achieves a VideoGen-Eval visual total score of 83.17, 0.41 points above the strongest distilled baseline. PDMD also achieves the best performance on all six audio metrics among the compared 4-NFE models. Qualitative comparisons and user studies favor PDMD over the distilled baselines in visual quality, motion, and audio quality. Code and models are available at https://pdmd2026.github.io/.