Closed-Loop Triplet Synergistic Generation for Long-Form Video
2026-06-15 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionMultimedia
AI summaryⓘ
The authors present CoTriSyGen, a new way to create long videos made of several clips that stay consistent in characters and story over time. Their approach uses a feedback loop that combines planned ideas, memory of what was generated, and the images themselves to check for mistakes and fix them. This method helps keep characters looking the same, maintains story continuity, and improves how well the generated video matches the initial descriptions. Tests show it works better than previous methods at keeping videos coherent and visually consistent across multiple shots.
multi-shot video generationstoryboard-driven pipelinevisual-text-memory synergyintra-shot refinementinter-shot refinemententity-centric memoryprompt engineeringvision-language modellong-form video coherenceiterative correction
Authors
Xinlei Yin, Xiulian Peng, Xiao Li, Zhiwei Xiong, Yan Lu
Abstract
Multi-shot long-form video generation remains challenging due to identity drift and compounding inconsistencies across shots. While storyboard-driven pipelines improve controllability, they are often executed in a feed-forward manner, with limited mechanisms to incorporate generated visual evidence back into subsequent conditioning. We propose CoTriSyGen, an agentic framework that formulates multi-shot long video generation as a closed-loop visual-text-memory synergy process, where planned intent, persistent memory, and generated visuals are jointly leveraged for iterative correction and long-range coherence. A vision-language-model-based analyzer reasons over this triplet and produces updates to both prompts and memory along two pathways: (i) intra-shot refinement, which triggers targeted regeneration when semantic or compositional violations are detected and refines image-to-video prompt for coherent motions; and (ii) inter-shot refinement, which rewrites subsequent-shot prompts to propagate newly manifested entities or attributes and improve prompt quality (e.g., compositional grounding and cinematic fluency) based on generated evidence. The loop is grounded in an entity-centric memory modeled as a mutable visual state that evolves as the story progresses, which is continuously updated by both the generator and the analyzer by adding new and evolved entities to reflect appearance changes, accumulated multi-view evidence, and multi-entity compositions. Experiments on our curated StoryBench benchmark demonstrate substantial improvements in cross-shot consistency, prompt adherence, and cinematic continuity over representative methods.