Video generation models struggle to create varied motion and camera effects
DiVid: Diagnosing Dimension-Specific Diversity Collapse in Video Generation Models
Computer Vision and Pattern Recognition
Summary
Video generation models often produce very similar videos even when asked to create different ones from the same description. The authors developed DiVid, a new way to measure how diverse these videos are across six different parts, like motion or scene. They found that models might seem good overall but actually fail to vary certain parts, especially how things move or where the camera is. Their work helps explain why these models lack variety and suggests ways to improve them.
What this means in practice
- •For video content creators: Identify which aspects of video generation models limit creativity by diagnosing diversity in motion and camera angles for richer video outputs.
- •For multimedia app developers: Improve video generation features by tracking and controlling diversity dimensions like motion and style for more varied generated videos.
Authors
Huanran Hu, Zihui Ren, Dingyi Yang, Zhinan Song, Guozheng Wu, Tiezheng Ge, Qin Jin
Abstract
Despite remarkable progress, video generation models often produce highly similar outputs when repeatedly sampled from the same prompt, limiting their usefulness for creative exploration. Existing diversity evaluations primarily rely on global scalar metrics, which obscure where diversity collapses in the spatiotemporal space of videos. We introduce DiVid, a dimension-level diagnostic framework that decomposes video generation diversity into six interpretable dimensions: Semantic, Style, Subject, Scene, Motion, and Camera. Each dimension is measured through a reproducible computer-vision pipeline and analyzed alongside quality and instruction faithfulness to examine potential trade-offs. Systematic evaluation of representative video generation models reveals that diversity is highly dimension-specific: models with strong global diversity scores still collapse on specific factors, particularly Motion and Camera. These rankings persist after filtering unfaithful generations, indicating genuine capability differences rather than off-prompt outputs. Beyond measurement, controlled prompt interventions identify two fundamental bottlenecks: default mode convergence, where models fall back to dominant patterns under open-ended prompts; and realization gaps, where models fail to faithfully realize diverse, explicitly requested alternatives, particularly for temporal factors. The larger faithfulness losses for temporal factors highlight the difficulty of controlling motion and camera variation through text alone. DiVid thus shifts the study of diversity from measuring whether it exists to diagnosing where and why it collapses, and provides actionable directions for dimension-aware training objectives and control signals. The framework will be released to facilitate future research on diverse and controllable video generation.