MotionSpec improves video generation with smoother realistic motion
MotionSpec: Spectral Trajectory Supervision for Motion-Consistent Video Generation
Computer Vision and Pattern Recognition
Summary
Making videos from text that show smooth and realistic movement is hard. The authors found that many generated videos look good frame by frame but have weird or jumpy motion over time. They created MotionSpec, which uses a special way to check and fix how motion changes in a video by looking at patterns in movement over time. This helps videos keep consistent and believable motion while still looking visually nice.
What this means in practice
- •For video game developers: Generate in-game video sequences with more consistent and smooth realistic motion from text descriptions for enhanced storytelling.
- •For advertising teams: Create high-quality promotional videos from scripts that maintain believable and fluid motion to improve viewer engagement.$Commercial implications: Enables selling automated video generation services that produce motion-consistent ads at scale to marketing companies.
Authors
Ziqi Ni, Rui Li, Shiqi Jiang, Wei Zhou
Abstract
Recent advances in text-to-video generation have enabled high-fidelity visual synthesis, yet realistic motion remains challenging. Generated videos may exhibit temporal discontinuities, inconsistent action progression, and structural distortions during complex movements. Even when individual frames appear realistic, the underlying motion may evolve in inconsistent or implausible ways. Standard generative objectives provide limited motion-specific supervision, leaving motion evolution insufficiently constrained. In this paper, we propose MotionSpec, a motion supervision framework centered on Spectral Trajectory Consistency (STC). STC constructs dense anchor-relative motion trajectories and transforms them into motion spectral volumes via a temporal Fourier transform. By aligning the spectral amplitude and phase of predicted and target trajectories, STC constrains both motion strength across temporal frequencies and the temporal organization of motion. To complement this trajectory-level supervision, we introduce Local Flow Consistency (LFC), which aligns consecutive-frame optical flow between predicted and target videos to stabilize local motion transitions. Experiments demonstrate that MotionSpec consistently improves motion consistency, temporal coherence, and plausibility while preserving visual fidelity.