Multimodal AI models evaluated for automated video ad clipping
AdShot: Benchmarking Multimodal Large Language Models for Video Advertisement Clipping
MultimediaComputer Vision and Pattern Recognition
Summary
Creating shorter versions of video ads is often slow and expensive for advertisers. The authors introduce AdShot, a test set to see how well advanced AI models can pick the right video shots to make these shorter ads. They tested many models and found smaller ones sometimes do as well as bigger ones, that models tend to make ads a bit too long, and that they prefer certain types of content over others. This helps us understand how well AI can assist in editing ads automatically.
What this means in practice
- •For advertising teams: Use the AdShot benchmark to evaluate AI tools for automating video ad trimming within specific duration requirements.
- •For video editing software developers: Incorporate benchmarked AI models to reduce manual work in video ad editing by selecting key shots automatically.$Commercial implications: This enables development of automated ad editing products that lower labor costs for marketing companies.
Authors
Wen Xie, Om Rastogi, Sai Siddhartha Vivek Dhir Rangoju, Gijs Overgoor, Yakov Bart, Sarah Ostadabbas
Abstract
Advertisers routinely create multiple duration variants of ads to meet marketing budgets and viewer preferences, a labor-intensive and costly process. While multimodal large language models (MLLMs) excel at general video understanding, their ability to perform specialized editorial tasks remains unexplored. We introduce AdShot, a benchmark for evaluating MLLMs on shot selection for ad clipping. It contains 823 professionally edited ad pairs (30-sec sources and 15-sec edits) across 194 brands and 17 industries, annotated with shot-level boundaries and content features (action, emotion, information, imagery). We evaluate 13 state-of-the-art video-only MLLMs and 2 audio-visual MLLMs (7B-72B parameters) on shot selection accuracy (Precision, Recall, F1), duration constraints (15s), and content preservation, including an audio ablation. We find: (1) model architecture matters more than scale, with smaller models often matching or exceeding larger ones; performance is strongest for target ads with 6-10 shots; (2) models exhibit a mean absolute duration error of 5.39s, with most tending to overshoot the 15s target, though they can still substantially reduce human editing effort; (3) all models exhibit systematic content biases, favoring information-rich and emotion-focused shots while under-selecting action and imagery. We publicly release our benchmark to facilitate research at the intersection of multimodal AI and computational advertising.