DUET: A Diversity-Quality Duet of Distillation Experts for Two-Step Video Generation
2026-08-10 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors study ways to make video generation faster without losing quality or variety. They find that two main methods each have trade-offs: one keeps videos diverse but lower quality, the other keeps quality but less diversity. They propose DUET, which combines two specialized models, each focusing on different noise levels to get both good quality and diversity. They improve this further with DUET+, using reinforcement learning to better coordinate the models. Their approach shows strong results in balancing video quality and variety in just two generation steps.
diffusion modelsvideo generationiterative samplingtrajectory-level distillationdistribution-level distillationnoise-level specializationreinforcement learningtwo-step generationmodel distillationquality-diversity trade-off
Authors
Zian Li, Litong Gong, Borui Liao, Pengfei Liu, Xinyu Wang, Xinyuan Wei, Yifan Gao, Tiezheng Ge, Muhan Zhang
Abstract
Diffusion models have enabled high-quality video generation in recent years, but the high cost of iterative sampling hinders their practical deployment. Few-step distillation alleviates this cost, yet exposes a quality--diversity trade-off between its two dominant paradigms: trajectory-level distillation (e.g., sCM) favors diversity, whereas distribution-level distillation (e.g., DMD) favors quality. Targeting extreme two-step video generation, we introduce DUET, which reconciles the two paradigms through a noise-level duet of experts: an sCM expert takes the high-noise step to lay out diverse structure, and a DMD expert takes the low-noise step to refine appearance detail. Since the two experts are trained independently with their native objectives, DUET sidesteps the optimization difficulties of loss-level combinations and delivers quality and diversity jointly rather than trading one for the other. We further identify the relay interface and the high-noise stage as the remaining bottlenecks, and address them with RL-guided expert adaptation, yielding DUET+. With the Wan2.1-T2V-1.3B backbone, DUET lifts the two-step quality of sCM close to the level of DMD while retaining nearly all of its structural diversity---about twice that of DMD---and DUET+ further improves overall quality while preserving this diversity advantage. Together, these results establish noise-level expert specialization as a simple, effective paradigm for reconciling diversity and quality in two-step video generation.