Autoregressive video diffusion improves long videos by reducing error drift
Clean Forcing: Drift-Resistant Autoregressive Video Diffusion with a Frozen Base
Computer Vision and Pattern RecognitionMachine Learning
Summary
Generating long videos frame by frame can cause errors to build up, making the video quality drop quickly. The authors find that these errors follow a predictable pattern that can be learned. They create a small extra model that fixes these mistakes while keeping the main video generator unchanged. This method improves video quality for longer clips using much less real video data than before.
What this means in practice
- •For video generation developers: Create longer, higher-quality videos by reducing error accumulation without retraining the entire model or requiring large video datasets.
- •For animation software teams: Integrate low-cost corrector modules to maintain video consistency over long sequences generated from text prompts.
Authors
Wenqing Wang, Joonghyuk Shin, Jonathan Tremblay, Chan Hee Song, Yun Fu
Abstract
Autoregressive (AR) video diffusion models generate streaming video of arbitrary length by conditioning each new chunk of frames on those already generated. However, they suffer from exposure bias, where a model trained on clean context must continue from its own imperfect outputs, and errors compound until quality collapses within seconds (drift). Existing methods either require cluster-scale base retraining, apply training-free inference corrections with inconsistent results, or train frozen-base correctors for sampling and cache errors. We instead find that drift is largely deterministic and therefore learnable, as holding the noisy state fixed while varying only the history reveals a counterfactual velocity gap that is ~95% systematic across noise seeds. Based on this observation, we introduce Clean Forcing, which trains a 5.9M-parameter LoRA corrector on the frozen base model to regress this gap, with a closed-loop stage that adds DAgger-style aggregation and a drift-contraction objective to expose the corrector to its own rollouts. Because the clean histories can be the base model's own single-shot generations, training needs no real videos, and the merged corrector adds no inference cost. In 50 s text-to-video generation, Clean Forcing reduces Δ-drift from +11.3 to +1.65 without real video and to -0.16 with 40 real clips, improving on the best published result of +3.57 with over 150x less data. Extensive experiments demonstrate that Clean Forcing outperforms prior methods in aesthetic quality and data efficiency with reduced drift for long-horizon video generation.