MotionInsight diagnoses object motion errors in generated videos
MotionInsight: Diagnosing Object Motion Deficiencies in Generated Videos
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Videos created by AI often have mistakes in how objects move, but most ways to check video quality focus on how good it looks or if the video matches a description. The authors made a collection of videos with detailed notes about object motion problems and then built MotionInsight, a tool that looks specifically at how objects move, not just the video frames. MotionInsight helps find problems with motion by analyzing movement patterns and gives clear scores and explanations about where motion goes wrong. This makes it easier to spot and fix motion mistakes in AI-generated videos.
What this means in practice
- •For video generation developers: Assess and improve motion accuracy in AI-generated videos by using MotionInsight’s diagnostic scores and explanations.
- •For quality assurance teams: Detect subtle motion defects in generated video content to ensure higher fidelity in video production pipelines.
Authors
Jiahao Zhan, Yongrui Ma, Qunliang Xing, Xuanyu Zhang, Jingqi Tong, Junlin Li, Li zhang, Shijie Zhao, Tianfan Xue
Abstract
Despite rapid progress in video generation models, they still exhibit obvious motion deficiencies, often manifested as incorrect object motion. However, most existing video quality evaluations focus on aesthetic quality or text-video alignment. To address this gap, we study object-centric motion fidelity assessment, evaluating target objects along object consistency, motion continuity, and physical plausibility. To achieve this, we first introduce VidMotion, a diagnostic dataset of 6,879 videos with designated moving objects and fine-grained annotations including dimension-wise scores and failure causes. We further propose MotionInsight, a diagnostic evaluator that shifts assessment from implicit RGB-frame observation to explicit motion-space diagnosis. By constructing motion-aware representations, MotionInsight makes subtle motion deficiencies more observable. We also introduce motion-specific rewards during GRPO to transform observed motion into a diagnostic assessment. Experiments demonstrate that MotionInsight provides an effective basis for diagnosing object motion deficiencies, producing human-aligned scores along three dimensions and grounded explanations.