RepForcing improves video generation quality with fewer steps
RepForcing: Representation-Space Initialization for Autoregressive Video Distillation
Computer Vision and Pattern Recognition
Summary
Generating high-quality videos quickly from complex models is challenging, especially with very few steps. The authors found that previous methods using intermediate video teachers still missed fine details in short generation schedules. They propose a new way called RepForcing, which trains video generation using internal representations instead of direct predictions, leading to clearer videos with fewer steps. Their method achieves better scores compared to earlier approaches, showing it can generate videos faster without losing quality.
What this means in practice
- •For machine learning engineers: Develop video generation tools that produce high-quality videos in fewer processing steps using the RepForcing initialization method.
- •For computer vision teams: Improve the efficiency of video synthesis in applications requiring fast generation by adopting representation-space supervision for model initialization.
Authors
Siming Fu, Zheming Fu, Lianrui Mu, Haoyang Huang, Chaojie Gu, Ruizhe He
Abstract
Distilling pretrained video diffusion models into few-step autoregressive (AR) generators typically combines student initialization with distribution matching distillation (DMD). An intermediate AR teacher can provide causally compatible targets for ODE initialization. However, causally compatible targets alone do not ensure high-quality generation under a short sampling schedule. We analyze the posterior-mean predictions favored by direct latent regression and empirically find that causal ODE initialization alleviates blur but still leaves fine detail unresolved. To address this limitation, we propose Representation Forcing, which learns few-step causal initialization through representation-space supervision rather than intermediate-teacher trajectory regression. The student is trained with teacher forcing by comparing clean predictions and training targets through normalized features of a frozen video diffusion transformer. This phase reuses the synthetic videos generated by a bidirectional model for AR teacher training and replaces both teacher training and causal ODE distillation, without generating additional causal ODE trajectories. The subsequent asymmetric DMD procedure remains unchanged. Empirical results show that, at four sampling steps, RepForcing better preserves generation quality than intermediate-teacher-based initialization before DMD. After DMD, our four-step chunk-wise model improves VBench Total from 84.04 to 84.66 (+0.62 points) and VisionReward from 6.326 to 10.074 (+59.25%) over the reported causal-ODE-based baseline.