Video generation method improves long sequences with full history access
Recast the Past: Stable Long Video Generation with Full Context Memory
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Generating long videos is challenging because computers need to remember everything that happened before, which becomes too difficult as the video gets longer. The authors present ReCam, a new method that groups parts of the past video into memory units, letting the model refer back to the entire history without slowing down. They also use a special technique to keep the video consistent over time, avoiding glitches or changes in appearance. Their approach outperforms other methods in quality and consistency for long videos.
What this means in practice
- •For video content creators: Produce longer, coherent video sequences where past details remain consistent throughout the clip.
- •For animation software developers: Incorporate improved memory handling to keep recurring characters and scenes visually consistent over extended animations.
Authors
Yukuo Ma, Cong Liu, Junke Wang, Junqi Liu, Haibin Huang, Zuxuan Wu, Xuelong Li
Abstract
Long-horizon video generation requires retaining an ever-expanding history, but conditioning on its growing token sequence is costly. Existing methods restrict or compress historical context, potentially losing information needed for future generation. This work proposes ReCam, which groups local spatiotemporal blocks of historical latents into vectors and projects them directly into the high-dimensional hidden space of a Diffusion Transformer. The model learns to use the resulting memory tokens through native self-attention, reducing token count while retaining block-level access to the full history. To mitigate temporal drift during autoregressive rollout, we further introduce dual-pointer denoising with separate noise schedules for recent context and target frames. Among the evaluated methods, ReCam achieves the highest VBench-Long overall score of 80.1 and competitive static-scene revisitation performance on R2M-Bench. It also leads on all four consistency metrics for off-screen subject reappearance, including clothing consistency of 4.09 versus 3.03 for the best baseline.