Self supervised training improves few step video diffusion models
Beyond Distribution Matching: Self-Supervised Representation Forcing for Few-Step Video Generation
Computer Vision and Pattern Recognition
Summary
Making computer models create videos quickly is hard because the models often lose detail or make flickering mistakes. The authors noticed that existing methods only focus on matching the final video output, which isn’t enough when making very short video prediction steps. They added a new kind of training that also teaches the model to match hidden internal clues, helping it keep finer details and smoother motion. This approach works with many kinds of video models and doesn’t need extra tools or much more computing power.
What this means in practice
- •For video content developers: Create short video clips with higher detail and fewer flickers using faster video generation models.
- •For animation studios: Improve motion quality and spatial accuracy in quickly generated animation sequences without extra hardware costs.
Authors
Junyi Chen, Haiyu Zhang, Zhoujie Fu, Tengfei Wang, Chunchao Guo
Abstract
Few-step distillation is essential for deploying modern video diffusion models. Among existing approaches, Distribution Matching Distillation (DMD) has emerged as a highly promising paradigm by achieving strong output-space fidelity without adversarial training, while remaining flexible across various student architectures. We observe, however, that DMD fundamentally relies on an output-space objective, lacking explicit regularization on the model's internal representations. As the distillation step count aggressively shrinks, this output-only supervision becomes insufficient, causing the distilled student to degrade in both fine-grained spatial structures and coherent temporal dynamics. To address this limitation, we introduce Self Flow Forcing (SF$^2$), which incorporates a predictive representation loss into the DMD framework without altering its output objective. By providing the student and an Exponential Moving Average (EMA) teacher with heterogeneous noise levels, we establish an information asymmetry. This formulation forces the student to predict the teacher's internal representations from a more corrupted input, effectively integrating self-supervised representation alignment into distribution matching. Extensive experiments demonstrate that SF$^2$ effectively mitigates temporal flickering and consistently improves both spatial fidelity and motion quality. Furthermore, the proposed mechanism is architecture-agnostic - seamlessly supporting both bidirectional and causal models - and requires no external feature extractors, introducing minimal computational overhead.