CraftTrace enables flexible editing of full videos using video structures
CraftTrace: Unflattening Videos into Malleable, Creation-Inspired Structures for Generative Editing
Human-Computer Interaction
Summary
Editing videos with many scenes and shots is hard because you must find and change parts one by one. The authors introduced CraftTrace, a tool that turns a whole video into a map of scenes, characters, and shots you can easily change. An AI helps apply your edits across the whole video so you don’t have to do repetitive work. This makes exploring ideas and refining edits faster and easier.
What this means in practice
- •For video editors: Edit entire multi-scene videos by manipulating structured video elements and letting AI apply changes consistently across shots.$Commercial implications: Supports creating advanced video editing products that streamline multi-shot edits for post-production studios.
- •For game developers: Prototype video sequences rapidly by restructuring scenes and character appearances with AI-driven generative editing tools.
Authors
Boyu Li, Yuqian Zhou, Duotun Wang, Ding Li, Zhe Lin, Nanxuan Zhao, Zeyu Wang, Lin-Ping Yuan, Hongbo Fu
Abstract
Recent generative video editing models enable video content modification (e.g., changing a character) but target short clips. Extending them to full multi-shot videos requires tedious work to locate relevant content across shots, segment it into clips, craft context-aware editing prompts for each clip, and repeatedly articulate complex editing intent. To address this, we explore an interaction paradigm for editing through underlying video structures (e.g., scripts, scenes, characters, shots, and their relationships). We present CraftTrace, an interactive prototype that transforms a video into a malleable, multilevel structure for generative editing. Users work in task-centric workspaces to modify elements or reshape relationships, while an AI agent translates and propagates changes across the video. A user study and expert review show that this structure helps users understand videos, formulate and refine editing intent, and explore alternatives, supporting rapid prototyping during early-stage exploration and full video post-production.