Ai agents struggle with real video editing in finished deliverables
Timeline-Bench: Evaluating Agents on Realistic Video-Editing Tasks, from Raw Footage to Final Cut
Computer Vision and Pattern RecognitionArtificial IntelligenceMultimedia
Summary
Video editing involves many complex steps, from choosing the right shots to assembling a polished final video. The authors created Timeline-Bench, a test set of 56 real video editing tasks, to see how well AI agents can complete these tasks from start to finish. They found that even the best AI agents only completed about one quarter of the tasks successfully, mostly failing the quality checks. Human editors preferred the original edits most of the time, showing that current AI still lacks creativity and finesse in video editing.
What this means in practice
- •For video production teams: Use Timeline-Bench to evaluate and improve AI tools tasked with automating parts of video editing workflows from raw footage to final product.
- •For media content platforms: Assess AI agent capabilities in video editing to inform deployment strategies for automated content assembly and quality assurance.
Authors
Gunin Gupta, Nirmit Arora, Pavan Kalyan Tankala
Abstract
AI agents increasingly carry out long-horizon professional work, but their evaluations rarely require a finished creative deliverable. To this end, we introduce Timeline-Bench, a benchmark of 56 real video-editing tasks, each asking an agent to turn raw production material into a finished video. Tasks range from selecting dialog takes and shaping interview footage into a story to cutting commercials from product shots, voiceovers and graphics. Every task provides a brief, source assets, a container and a set of tests. A task is resolved when the output passes every test. The tests check the delivery format, the content and the brief's explicit requirements, and include a quality test calibrated on 2,582 blind judgments by 43 video editors. We evaluate 16 agents that pair frontier models with coding-agent harnesses such as Codex, Claude Code and OpenCode. The best, GPT-6 Astra in Codex with curated editorial guidance, resolves only 15 of the 56 tasks (26.8%), and the average agent resolves 14.0%. Human editors prefer the reference edit in 83.5% of judgments. Most unresolved runs (562 of 771) fail only the quality test: agents perceive footage through stills and transcripts and check their renders for defects, not craft. We release the tasks, verifier and per-run results at https://timelinebench.tensortest.com.