Video diffusion model reveals hidden 3D human motion signals
Parasitic Co-Denoising: Unlocking 3D Human Motion Generation in a Frozen Video Diffusion Model
Computer Vision and Pattern Recognition
Summary
Creating 3D human motion from videos usually needs special training, but the authors found that a large video generation model already contains usable motion information inside it. They developed a new method that extracts 3D motion directly from the model's inner steps without changing it. This approach produces both video and 3D motion together and matches or beats dedicated motion models while being much smaller and more efficient.
What this means in practice
- •For animation studios: Create synchronized 3D human motion and video clips from text prompts without training separate motion models.$Commercial implications: Enables production of text-driven 3D motion and video content efficiently for media and entertainment markets.
- •For game developers: Generate paired realistic human motion and accompanying video footage for use in character animation pipelines without costly motion capture.
Authors
Yunjiao Zhou, Junlang Qian, Lihua Xie, Jianfei Yang
Abstract
Despite never being supervised on explicit 3D motion, large-scale text-to-video diffusion models synthesize realistic human motion in their generated videos. We ask whether this implicit knowledge can be turned into explicit 3D motion generation, without training a separate motion model. Probing a frozen Wan2.1 reveals that a recoverable motion signal is present in its intermediate states across the entire denoising schedule, not confined to the clean output. Motivated by this, we introduce parasitic co-denoising, a paradigm in which motion is decoded from the host model along its denoising schedule rather than produced by an independent generator. We instantiate it as the Parasitic Motion Decoder (PMD), an efficient flow-matching decoder that shares the host's noise schedule and reads its intermediate features through a $σ$-adaptive multi-layer fusion, leaving the host unmodified. Drawing its coverage from the host rather than from motion data, PMD leads dedicated motion generators on text-motion alignment at a small fraction of their trainable parameters, while producing paired video and motion in a single pass that motion-only baselines cannot match.