Memory forcing improves video streaming with longer event recall
Memory Forcing: Attendable Mid-Horizon History for Streaming Video Generation
Computer Vision and Pattern Recognition
Summary
Generating videos frame-by-frame can forget important events that happened in the middle of a clip because it only remembers the start and most recent frames. The authors introduce Memory Forcing, which keeps intermediate events in memory without using more space, so the video can remember things better as it plays. They divide memory into parts that store past important events and recent action, letting the system recall scenes and subjects even after a long time. This approach helps create longer, more realistic streaming videos that keep track of what happened earlier.
What this means in practice
- •For video streaming engineers: Build streaming video systems that generate longer clips while maintaining recall of intermediate events without increasing memory size.
- •For visual effects developers: Create more coherent and realistic dynamic videos by preserving scene and subject consistency over time in generative models.
Authors
Jiaming Zhang, Xinyu Wang, Huafeng Shi, Gangshan Wu, Limin Wang
Abstract
Autoregressive video diffusion enables causal video streaming without a bidirectional pass over the full clip, but existing few-step systems usually retain only the opening and most recent frames in a fixed-size KV cache. Once an event leaves this window, later frames can no longer attend to it, a failure we term mid-horizon forgetting. We present Memory Forcing, a few-step streaming method that preserves this missing history without increasing the cache size. Its Archive \& Working Banks partition the cache into sink, archive, and working regions, retaining diverse intermediate events alongside recent motion under fixed memory. Because absolute temporal indices drift outside the training range, Bank-aware RoPE reassigns indices at attention time so each bank remains distinguishable. At 1.3B, Memory Forcing leads on longer clips, shows the smallest drop from 5s to 60s among methods reporting all four lengths, and preserves subjects and scenes through leave-and-return. The same design scales to Wan2.2 5B, producing more physically plausible, realistic, and dynamic videos and, to our knowledge, the first public 5B model on this forcing line.