Framework improves consistency in long videos using complementary prompts
Complementary Retrieval-Augmented Prompting for Consistent Long-Form Video Generation
Computer Vision and Pattern Recognition
Summary
Generating long videos with consistent characters and scenes is tough because each video segment is often created separately without enough shared context. The authors show a way to pick just the right past video parts as references, matching exactly what’s needed for the next segment to keep things consistent. Their method organizes story details like characters and objects, then fetches complementary visual pieces to guide the video generator without needing to retrain it. This leads to better continuity across the whole video and more control over the story.
What this means in practice
- •For film production teams: Maintain consistent characters, scenes, and objects across independently generated video segments in automated storytelling workflows.
- •For advertising agencies: Create coherent long-form promotional videos by combining fine-grained visual references to keep branding elements stable throughout the content.
Authors
Xianghan Wei, Xiaoda Yang, Zhi Wang, An Pan, Daoan Zhang, Huayi Zhang, Yan Zhang, Wei Xu, Zishun Liao, Jianwen Lou
Abstract
While recent video foundation models excel at generating high-quality short videos, long-form video generation remains a critical challenge, where a major bottleneck lies in conditioning independently generated shots to preserve consistent characters, scenes, and objects throughout a story. Existing training-free approaches typically condition target shots using retrieved historical visuals. However, these references often suffer from severe informational mismatch, either introducing irrelevant contextual redundancy or failing to provide the full combination of required elements for the target shot. To resolve this, we present Complementary Retrieval-Augmented Prompting, an agentic framework that strategically aggregates a compact set of mutually supportive historical references to achieve complete and targeted conditioning for long-form video generation without retraining or modifying the underlying generator. Specifically, our framework explicitly models the visual elements required by each target shot by parsing the narrative script into a text-grounded visual element registry that tracks characters, objects, scenes, and their shot-level states. A VLM-annotated keyframe library further maps these elements to past visual observations. Guided by the required elements, our agent retrieves complementary references that maximize target-element coverage while minimizing historical noise. Finally, the retrieved references, structured element states, and grounding instructions are assembled into a unified prompt for the frozen video generator. This element-aware process provides comprehensive conditioning while remaining fully interpretable. Quantitative and qualitative evaluations on multi-shot story generation demonstrate that our method consistently outperforms recent-frame, memory-based, and entity-level retrieval baselines in cross-shot consistency and text-controllability.