Video prompt inversion benchmark reveals current model limitations
VI-Bench: Benchmarking Prompt Inversion from AIGC Videos
Computer Vision and Pattern Recognition
Summary
Understanding the exact text prompts used to create AI-generated videos is important for editing and reusing those videos. Existing tests only check if a caption describes a video, but don’t measure if the key controls or instructions can be recovered to recreate it. The authors created a new large benchmark named VI-Bench that tests how well different AI models can reverse-engineer these prompts from videos. Their results show that even the best models struggle, especially with complex videos needing detailed control.
What this means in practice
- •For video editing teams: Identify and recover original prompt controls from AI-generated videos to enable precise video editing and creative reuse.
- •For content moderation teams: Assess risks of prompt leakage by detecting if AI-generated videos reveal sensitive or proprietary prompt details.
Authors
Wulin Xie, Rui Zhao, Kecen Li, Xiujin Liu, Bokang Zhang, Zheng Liu, Xinwen Hou, Chen Gong
Abstract
Recent advances in video generation have made prompt-based control increasingly central to AIGC video generation. Prompts specify what a video should depict and how it should be represented, controlling factors such as visual style or camera behavior. Understanding this recoverability is important both for creative reuse and editing, and for assessing prompt leakage risks. However, existing video understanding benchmarks do not measure this capability: a caption may describe what is visible, but a replayable prompt must recover the generation-relevant controls needed to reproduce the video. To address this gap, we introduce VI-Bench, a benchmark built from 16.1 million real-user prompts and 900 human-verified AIGC videos. VI-Bench spans three progressively harder settings, namely single-shot semantic grounding, control over style and camera behavior, and multi-shot compositional inversion, and evaluates five generation-critical dimensions: subject, action, scene, style, and camera. We evaluate 18 representative VLMs, including 2 proprietary and 16 open-source models on VI-Bench, using an Inversion Score that measures prompt-level alignment with the original prompt and video-level fidelity of the regenerated video. The results reveal substantial limitations: even the strongest model achieves only 0.632 on Inversion Score, performance degrades sharply as samples require richer control and multi-shot reasoning, and models often produce plausible prompts whose regenerated videos deviate from the reference. These findings show that video prompt inversion is a distinct and under-evaluated capability requiring models to transform visual understanding into replay-stable generative control.