Uncertainty dmd improves diversity in autoregressive video generation
Uncertainty DMD: Restoring Diversity in Few-Step Autoregressive Video Distillation
Computer Vision and Pattern Recognition
Summary
Generating videos step-by-step with computers can become repetitive, producing similar-looking results even when starting with different random inputs. The authors found that a common technique, Distribution Matching Distillation (DMD), causes this problem by losing uncertainty early on and repeating it throughout the video. They propose a simple fix called Uncertainty DMD, which adds randomness back in at key points during both training and generation. This approach helps produce more varied and lively videos without sacrificing quality.
What this means in practice
- •For video generation engineers: Enable more diverse and natural motion in autoregressive video models during both training and inference without changing model architecture.
- •For game developers: Produce more varied and dynamic AI-generated video content or animations that better retain motion diversity when running on resource-limited setups.
Authors
Zixuan Duan, Xunzhi Xiang, Yabo Chen, Xin Zhang, Changhan Liu, Haibin Huang, Chi Zhang, Qi Fan, Xuelong Li
Abstract
Few-step distillation improves the efficiency of autoregressive (AR) video generation, but often causes diversity collapse: under the same prompt, different noise samples tend to produce highly similar videos with weakened motion dynamics. We analyze this degradation in Distribution Matching Distillation (DMD)-distilled AR video generators and find that, in the autoregressive setting, it takes the form of a structured uncertainty collapse: the mode-seeking bias of DMD maps different noise samples to nearly identical first chunks, and the deterministic AR cache then propagates this collapsed state to all subsequent chunks, turning a local loss of stochasticity at the rollout root into a global suppression of temporal variation. Based on this analysis, we propose Uncertainty DMD, a simple uncertainty-injection framework that restores stochasticity at two key stages of AR generation: a timestep perturbation for the first chunk to increase first-chunk diversity, and a stochastic cache-writing mechanism for later chunks to preserve uncertainty in autoregressive conditioning. The method requires no architectural changes and introduces only lightweight perturbation operations. The same perturbation mechanisms are used during both training and inference. Experiments show that Uncertainty DMD consistently improves diversity and motion dynamics while maintaining comparable per-sample visual quality.