Multi shot video generation struggles with professional editing techniques
Beyond Coherence: Benchmarking Professional Editing-Technique Execution in Multi-Shot Audio-Video Generation
Artificial Intelligence
Summary
Making videos with multiple shots that follow professional editing rules is hard, even for advanced AI systems. The authors found that current models often create videos that look okay but don’t follow detailed editing instructions like smooth transitions or precise cuts. They made a new test called CutCraft that checks if AI-generated videos really use typical editing tricks. Their tests showed many models fail to do these editing techniques well, even if the videos seem coherent and watchable.
What this means in practice
- •For video production teams: Evaluate AI tools for multi-shot video editing to ensure they follow professional editing rules like transitions and cuts.
- •For media software developers: Use CutCraft benchmark to improve AI video generators for better shot planning and accurate editing technique execution.$Commercial implications: This paper enables building AI video editing software with reliable professional-level editing features attractive to content creators.
Authors
Tianyi Zeng, Junchao Liao, Yujie Wei, Ziying Zhang, Litao Li, Tianyi Wang, Zhichao Wei, Shuyao Xu, Wenwen Qiang, Siyu Zhu, Zhenghao Zhang, Long Qin
Abstract
Recent multi-shot audio-video generators can produce increasingly coherent and cinematic outputs, but coherence does not imply the ability to execute editing techniques. Professional editing depends on shot structure, transition grammar, audio-video cut relations, and montage, yet existing benchmarks largely rely on proxies such as content quality, synchronization, or physical plausibility, systematically missing whether such editing instructions are actually executed. We introduce CutCraft, the first benchmark for editing-technique execution in multi-shot audio-video generation. CutCraft extends structured multi-shot prompts with explicit editing specifications and is paired with a hierarchical hybrid evaluation framework that combines shot-structure alignment, expert-model metrics, tool-grounded multimodal judgment, and rubric-based question answering. Beyond evaluation, we design an agentic editing baseline that decomposes generation into planning, shot-level synthesis, and post-hoc composition, explicitly realizing editing semantics such as J-cuts, L-cuts, and transition timing. Across 13 state-of-the-art closed- and open-source models, CutCraft reveals a consistent gap between coherence and editing-technique execution: current systems often produce plausible multi-shot videos yet fail to execute editorial instructions reliably. We find unstable shot structures, weak control of transition execution, and sharp degradation on higher-order montage, while aesthetic quality is only weakly correlated with editing-technique compliance. The benchmark and metrics, and the editing agent baseline are available at https://github.com/AlibabaResearch/cut-craft-bench.