Weave Forcing improves memory use for consistent long interactive videos
Weave Forcing: Compositional Memory Routing for Interactive Long Video Generation
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Creating long videos where scenes keep making sense is hard, especially when different shots mix characters and backgrounds. The authors present Weave Forcing, which uses a smart way to pick and reuse parts of earlier scenes based on what the user wants, like specific characters or backgrounds. This method helps the video tool remember important details without mixing in irrelevant content, keeping the story consistent and visually clear over time. They also created techniques to avoid glitches when references are partial or incomplete. Tests show this approach keeps characters and backgrounds steady across scenes while still matching the text prompts closely.
What this means in practice
- •For interactive media developers: Enable software to generate long videos that blend characters and settings from past scenes more consistently during user-driven storytelling.
- •For game developers: Improve cutscene creation tools to maintain visual and narrative consistency by selectively reusing scene elements over extended sequences.
Authors
Ziyi Wang, Junchi Yao, Heqian Qiu, Wenbo Shi, Chengjiu Wang, Jinyang He, Binkai Hong, Hongliang Li
Abstract
Recent advances in autoregressive video generation have improved temporal consistency over extended durations, yet interactive storytelling requires more than continuous scene extension: a new shot may combine characters and backgrounds from different historical shots. Whole prompt retrieval can overlook the distinct reference needs of individual components, while directly combining all historical memories may introduce unrelated visual content. To address these problems, we present Weave Forcing, a training-free framework for compositional memory reuse in interactive long video generation. First, we use an LLM for semantic slot routing to decompose user prompts into character and background descriptions and explicitly select suitable historical references for each component. To isolate the required content, masked memory weaving uses contrasting attention maps conditioned on semantic slots to construct refined semantic masks, selectively exposing relevant tokens from compressed historical KV memories to guide the generation of the current shot. We further introduce coverage adaptive RoPE to adjust temporal offsets and memory retention according to no, partial, or full reference coverage, addressing visual artifacts observed when incomplete historical references are positioned close to the current generation. Extensive experiments demonstrate that Weave Forcing improves cross-shot subject and background consistency while maintaining competitive visual quality and text alignment.