Motion Beyond Morphology: Bootstrapping Cross-Category Motion Transfer from Abstract Motion Representations
2026-08-03 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors focus on a way to animate a target video object by copying movement from a reference video, even when the two objects look very different. They propose a two-step method: first, learning general motion patterns that apply across different shapes, and second, using these learned patterns to directly generate new animations without extra motion calculation. They also created datasets and tests to check how well their method works with objects that are similar or very different. Their experiments show improved motion quality and better preservation of the target object's appearance.
video motion transferstructural correspondencemorphologymotion dynamicscross-category animationmulti-granularity motionvideo generationmotion fidelitymotion datasets
Authors
Zhixue Fang, Zhimin Zhang, Bi'an Du, Zijie Meng, Yan Zhou, Wei Hu, Guoxin Zhang, Pengfei Wan, Kun Gai
Abstract
Video motion transfer aims to animate a target object using dynamics from a reference video. Existing formulations largely rely on fixed structural correspondence, which becomes ill-defined when reference and target objects differ substantially in morphology, articulation, or deformation mechanisms. We introduce Motion Beyond Morphology, a perspective that seeks to transfer motion beyond fixed structural correspondence, by preserving dynamics that remain meaningful across different target morphologies. To realize this, we propose a two-stage framework. Stage~I learns complementary multi-granularity abstract motion views and uses them to bootstrap cross-category video pairs that preserve transferable dynamics across diverse morphologies. Stage~II internalizes this supervision into direct reference-video-conditioned generation, removing the need for explicit motion extraction at inference. We further introduce OpenVMT-Dataset and OpenVMT-Bench for training and evaluating image- and text-conditioned motion transfer across Same, Near, and Far category gaps, and plan to release both upon acceptance. Extensive experiments demonstrate state-of-the-art motion fidelity and target preservation. Project page: https://miniz233.github.io/MotionBeyondMorphology/