Direct forcing improves few-step streaming text to video generation
Direct Forcing: A Single-Stage Training Recipe for Few-Step Autoregressive Text-to-Video Generation
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Generating videos step-by-step based on text descriptions can be slow if done in many steps. The authors propose a method called Direct Forcing that trains a video model in a single step to predict future frames quickly and accurately. It learns both how to continue a video from past frames and how to make the generated videos similar to real ones, using a balanced training approach. This method speeds up training and improves video quality without complicated setups.
What this means in practice
- •For video streaming developers: Create low-latency streaming video generators that quickly predict future frames from text inputs using a single-stage training approach.
- •For multimedia content creators: Generate short videos from text prompts with improved quality and faster training, enabling more efficient content production workflows.
Authors
Haojun Xu, Jie Huang, Xin Lu, Mingchen Zhong, Zihao Fan, Linjiang Huang, Si Liu
Abstract
Few-step autoregressive video generation enables low-latency streaming, yet prevailing training pipelines separate causal adaptation from distribution matching. Recent representation-based text-to-video methods retain this separation by starting from ODE-initialized checkpoints, even when the target distribution is defined directly by reference videos. We introduce Direct Forcing, a single-stage framework that uses reference videos both as paired examples of how a history continues and as a target population for self-generated rollouts. Causal Continuation Alignment learns paired spatiotemporal correspondence under reference histories, while Rollout Distribution Matching uses maximum mean discrepancy to align the representation distributions of self-generated and reference videos. Together, these complementary signals allow a pretrained video model to learn causal continuation and the target video distribution in a single training stage. Dynamic Gradient Balancing adapts the relative gradient contributions of continuation alignment and distribution matching during training. Starting from bidirectional Wan2.1-1.3B, Direct Forcing learns a four-step autoregressive generator in 250 updates without a separate initialization stage or online diffusion-score supervision, achieving 84.63 VBench Total and 86.01 Quality. Ablation studies further demonstrate the benefits of dynamic gradient balancing and reveal distinct representation preferences: deeper features benefit continuation alignment, whereas intermediate features yield better performance for rollout distribution matching.