Cleanmdm improves 3d motion capture data cleanup with multimodal guidance
CleanMDM: Clean Motion Diffusion Model for Multimodal Motion Cleanup
Computer Vision and Pattern Recognition
Summary
Motion capture data often has errors like missing parts, jitter, or unnatural movements, which makes it hard to use directly. The authors introduce CleanMDM, a tool that fixes these issues by using different types of clues like noisy 3D motion, sparse keyframes, and text instructions to guide the cleanup. Unlike previous methods that treat cleanup like a black box, CleanMDM lets users control the process more precisely and improves realism by reducing artifacts. It works better than earlier tools on multiple datasets and supports combining different inputs for flexible cleanup.
What this means in practice
- •For 3d animators: Use CleanMDM to automatically fix and refine noisy motion capture data with user guidance from sparse keyframes or text.
- •For game developers: Integrate CleanMDM to clean and improve in-game character animations from imperfect motion capture sources.
Authors
Zhe Li, Shicheng Wang, Bowen Cai, Huan Fu
Abstract
Motion capture data is rarely directly usable, as they typically exhibit missing segments, jitter, drift and contact artifacts. Traditionally, corrupted motions are cleaned by animators through the manual identification of keyframes from noisy motion, subsequent keyframe correction, and interpolation between corrected keyframes to reconstruct coherent motion. While the rise of generative motion models has made automatic cleanup feasible, most approaches operate as black box denoisers with limited controllability, making it difficult to preserve reliable segments or enforce specific user intents. Inspired by animation workflows, we present CleanMDM, a unified multimodal motion cleanup framework that formulates cleanup as masked conditional generation with plug-and-play conditions. This single model supports arbitrary combinations of noisy 3D motion, sparse 2D keyframes, sparse 3D keyframes, and text. This design enables both automatic cleanup without additional user annotation and controllable cleanup under multimodal guidance. To further improve motion realism, we incorporate the Latent Motion Quality Discriminator (LMQD) to better match kinematic distributions and reduce skating, jitter, and interpenetration artifacts, and we apply Mesh-Aware Contact Projection as a test-time optimization step to enhance contact and physical consistency. Experiments across multiple datasets demonstrate that CleanMDM consistently outperforms prior cleanup and generation baselines, and that low cost conditions (text and 2D keyframes) provide reliable controllability gains in multimodal cleanup scenarios.