Motioncoach improves fine professional motion editing in videos
MotionCoach: Constructing Multimodal Guidance for Video Professional Motion Editing
Computer Vision and Pattern RecognitionArtificial IntelligenceMultimedia
Summary
Editing videos to show expert movements in sports or martial arts is hard because specific techniques are complex and often lack detailed instructions. The authors created MotionCoach, an AI assistant that learns to guide video editing models by providing clear examples and instructions for these motions. This system learns from data and feedback to better understand professional motions and helps video editing tools make accurate and realistic changes without needing to retrain them. They also created a new benchmark to test how well the system works on many professional motion editing tasks.
What this means in practice
- •For video production teams: Enhance professional motion edits in sports and performance videos by supplying precise motion guidance without retraining underlying editing models.
- •For animation studios: Integrate multimodal professional motion prompts to improve realistic motion effects in character animation workflows.
Authors
Siyi Qian, Hanqing Yang, Weihuang Lin, Zihao Yang, Yichen Lu, Yuanhong Zheng, Sheng Li
Abstract
Video editing models achieve strong performance in modifying appearance, backgrounds, styles, and motions, empowered by large-scale training data. Beyond general scenarios, professional motions are significant in performance, sports, and martial arts. This paper focuses on professional motion editing, which involves fine-grained motion technique and temporal structure. Given professional motion names but lacking detailed descriptions, existing models often fail to generate professional motions, such as incorrectly lifting the front wheel under a Stoppie instruction. To address this challenge, we introduce MotionCoach, a trained multimodal agent that uses multi-turn tools and performs self-verification, producing a grounded prompt and a reference clip as reliable and sufficient multimodal guidance to inject external professional motion knowledge into video editing models. To supply training data, we build a data pipeline to construct trajectories. We cold-start the agent on MotionCoach-SFT-10k and apply Group Relative Policy Optimization (GRPO) on MotionCoach-RL-2k with rewards on the motion guidance prompt, the motion outcome video, and the format. For evaluation, we introduce VPMEBench, comprising 314 professional motion editing tasks with fine-grained references and annotations. Extensive experiments show that MotionCoach improves the overall score from 55.1 to 65.6 on Seedance 2.0 and from 51.2 to 64.5 on MiniMax-H3. Learning to construct multimodal guidance enables precise professional motion editing without retraining video editing models.