Multi-agent system improves autonomous video editing with scripting and critique
Auto-Director: Leveraging Multi-Agent Collaboration for Autonomous Video Editing
Computer Vision and Pattern Recognition
Summary
Editing videos automatically from lots of different clips is hard. The authors created Auto-Director, a system that works like a team of helpers: one plans the story, another organizes video clips, a third puts clips together, and a fourth checks and improves the final video. This approach helps make videos that follow a clear story better and look more professional than older methods. They also made a new way to measure how good these edited videos are.
What this means in practice
- •For video production teams: Automatically generate edited videos from unstructured footage with improved narrative and stylistic coherence, enabling faster content creation.
- •For marketing teams: Create polished promotional videos from diverse media sources without professional editing, ensuring alignment with branded messages and pacing.
Authors
Yuxia Chen, Wenbo Zhu, Yangguang Ji, Zhenxiang Jiang, Teng Zhong, Yu Zhang
Abstract
Autonomous video editing from open-world resources remains a challenging problem in multimedia content creation. Retrieval-based methods are constrained by fixed source material and lack creative flexibility, while generative models struggle with long-horizon coherence and often diverge from user intent. Both paradigms fail to replicate the structured, multi-stage reasoning that underlies professional post-production. To address these limitations, we present Auto-Director, a tool-augmented multi-agent framework that mirrors professional editorial workflows: a Planning Agent decomposes abstract topics into fact-grounded scripts via retrieval-augmented planning; a Material Agent indexes open-world footage through shot segmentation and visual tagging; an Editor Agent assembles semantically aligned clips with MLLM-driven spatial adaptation; and a Critic Agent governs an iterative Reflect-and-Revise loop that checks scene-level alignment and pacing. To rigorously evaluate this new paradigm, we introduce VEdit-Bench, a benchmark of 100 expert-curated topics with human-annotated Golden Scripts and a five-dimensional metric suite covering Narrative Logic, Factual Accuracy, Semantic Alignment, Compositional Quality, and Stylistic Cohesion. Extensive experiments demonstrate that Auto-Director significantly outperforms baseline methods in both automated metrics and human evaluation, effectively bridging the gap between amateurs and professional-grade video production.