Video dynamics editing separates motion from appearance for flexible control
Reimagine Video Dynamics
Computer Vision and Pattern Recognition
Summary
Most video editing tools let you change how a video looks but not how things move in it. The authors created a way to capture and edit the motion in a video separately from its appearance using a special token called a dynamics token. This token is learned by training a system to recreate a whole video from just the first frame and the token, so it focuses on how the scene changes over time. Their method lets you edit or retime video motions while keeping the original look, and it can understand instructions given in language to change the motion.
What this means in practice
- •For video game developers: Modify character and environment motions in game scenes independently from their appearance to create varied animations without re-rendering all visuals.
- •For advertising agencies: Adjust the timing and dynamics of product demonstration videos without changing the visual style to better fit campaign needs.$Commercial implications: This enables creating multiple dynamic versions of ads efficiently, offering tailored visual storytelling for different audiences.
Authors
Yu Yuan, Yawen Lu, Guoxian Song, Kevin Duarte, Ratheesh Kalarot, Di Chang, Xijun Wang, Stanley H. Chan
Abstract
Most video editing methods focus on changing the appearance of the source video, while offering limited control over its dynamics. We introduce Reimagine Video Dynamics (RVD), a framework that disentangles a compact, editable dynamics token from visual context. We learn this token through self-supervised reconstruction: given the first frame as visual context, a renderer must recover the original video from the dynamics token, encouraging it to capture how the scene evolves rather than how it looks. This disentanglement allows video dynamics to be edited directly while preserving visual context. We develop a language-guided dynamics-token editor that transforms source dynamics into target dynamics, and train it with a scalable counterfactual video-pair pipeline and a two-stage training strategy. Extensive experiments show that RVD enables effective video dynamics editing, training-free retiming, and appearance-controlled re-rendering.