StoryEngine improves coherence and consistency in video storytelling
StoryEngine: A State-Grounded Agentic Framework for Video Storytelling
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Making long videos that tell a story without mistakes or confusing parts is hard. Existing methods sometimes lose track of important details or change things unexpectedly as the video goes on. The authors of this paper created StoryEngine, a new system that keeps a clear record of what is happening in the story and where everything is in each scene. This system helps keep the story consistent and the visuals accurate by carefully managing how scenes and characters change over time. Tests show that StoryEngine makes better stories that are easier to follow and look more consistent than other methods.
What this means in practice
- •For game developers: Create more coherent and visually consistent cutscenes in video games by managing story states across scenes.
- •For animation studios: Automate the production of long-form animated videos with improved narrative flow and fewer visual mistakes.$Commercial implications: Enables production studios to sell automated tools that maintain story consistency and reduce manual correction in animation workflows.
Authors
Yingrui Wang, Zeqing Wang, Yeying Jin
Abstract
Despite recent progress in agentic multi-shot video generation, producing coherent and consistent long-form stories remains challenging. Existing agentic pipelines typically rely on textual shot plans or previously generated pixels, yet lack an explicit mechanism for propagating the consequences of story events and maintaining the video world state across shots. As a result, missing visual details may be reconstructed inaccurately, while visual drift may propagate across subsequent shots, undermining both narrative coherence and visual consistency. To address these challenges, we propose StoryEngine, a state-grounded agentic framework for video storytelling. StoryEngine establishes a separation between authoritative semantic plans and unreliable visual observations. Specifically, StoryEngine maintains a structured representation of entity placement and story-relevant states, and propagates event-induced changes to define the intended start and end states of each shot. To visually realize these states, StoryEngine constructs canonical references for recurring entities and environments, and compiles state and visual constraints into executable render plans. Meanwhile, to realize these states correctly, a bounded evaluation-guided repair loop further corrects local state inconsistencies. Together, these mechanisms preserve causal story progression and prevent local visual errors from propagating across shots. To comprehensively evaluate long-form storytelling, we construct a benchmark across diverse scenarios and visual styles, with metrics assessing storytelling quality, narrative coherence, and visual consistency. Experimental results demonstrate that StoryEngine consistently outperforms state-of-the-art methods across all evaluation dimensions, validating its effectiveness for coherent and consistent video storytelling.