Video flow models improved with decoupled advantage weighting for better motion quality
VideoAWR: Reinforcing Video Flow Models with Decoupled-Timestep Advantage-Weighted Rectification
Computer Vision and Pattern Recognition
Summary
Generating videos with AI often struggles to balance making videos look good while learning the right things step-by-step. The authors found that current methods either lead to blurry videos or unstable training. They propose VideoAWR, which separates how the model explores video possibilities from how it learns from them, keeping training stable and improving video quality. Their approach also makes generating videos faster and works well with existing guidance methods.
What this means in practice
- •For video generation teams: Improve training stability and visual quality of AI-generated videos without relying on classifier guidance during inference.
- •For animation software developers: Integrate better flow models that produce smoother motion and richer video details for generative animation tools.
Authors
Zheming Fu, Wenbin Ning, Ruizhe He, Wei Shang, Xiaoxiao Ma, Lianrui Mu, Chang Liu, Lei Wang, Siming Fu
Abstract
Regression-based RL for flow models mandates sampling \emph{without} classifier-free guidance (CFG) to align the rectification target with the reference marginal velocity. Extending Advantage-Weighted Rectification (AWR), which dynamically scales this regression target based on reward, to video generation encounters a cold-start pathology: unguided rollouts collapse into low-fidelity regions where intra-group reward gaps remain wide but no longer track perceptual quality, rendering the optimization target arbitrary. Conversely, utilizing guided rollouts to restore fidelity, while keeping the regression target tied to the guided sampling distribution, provably triggers a geometric guidance blow-up under exponential-moving-average (EMA) updates. We propose \textbf{VideoAWR}, which resolves this tension by untying sampling from the optimization anchor. An EMA reference policy generates \emph{guided} rollouts to restore reward separability, while the trainable policy is rectified against the \emph{unguided} conditional anchor. We prove this decoupled objective is mathematically exact: the centered advantage cancels the anchor in expectation, filtering out the reward-neutral guidance offset and retaining only the reward-correlated covariance increment. Consequently, inference runs single-branch and CFG-free, yet inference-time CFG preserves its original semantics. Furthermore, we introduce \emph{Decoupled Timestep Rectification}, supervising each rollout at an independently sampled timestep without bias to improve sample efficiency. Experiments on Wan2.1-1.3B and 14B demonstrate stable optimization and superior visual and motion quality in \emph{both} CFG-free and CFG-enabled regimes. Code is available at \pdfstartlink attr{/Border [0 0 0]} user{/Subtype /Link /A << /S /URI /URI (https://github.com/Fuzm7/FlowAWR) >>}% \textcolor{pearDark}{VideoAWR}\pdfendlink.