Motionspaceflow improves text-driven human motion generation and control
MotionSpaceFlow: Representation-Aware Flow Matching in Direct Motion Space
Computer Vision and Pattern Recognition
Summary
Generating realistic human motion from text can be tricky because current methods often rely on simplified spaces that limit detail and direct control. The authors introduce MotionSpaceFlow, a technique that creates detailed human motions directly in the full motion space without needing to compress or reconstruct data. Their approach also adapts noise and attention mechanisms based on the type of motion information, allowing better quality and control. This method lets users adjust specific joints or frames on the fly, even if the system wasn’t trained for that exact control.
What this means in practice
- •For game developers: Generate detailed and controllable human animations from natural language descriptions without retraining models for each control constraint.
- •For virtual reality creators: Create more realistic and editable avatar motions directly from text commands with precise control over individual frames and joints.
Authors
Qing Yu, Kent Fujiwara
Abstract
Recent advances in diffusion and flow models have substantially improved text-driven human motion generation. Yet most methods generate in low-dimensional, temporally downsampled latent spaces learned primarily for reconstruction, a bottleneck that can limit generation quality and preclude direct manipulation of individual frames and joints. We introduce MotionSpaceFlow (MSFlow), a representation-aware flow-matching framework that predicts clean motion directly in continuous motion space without a learned encoder or decoder. To account for the anisotropic structure of direct motion representations, we propose representation-aware noise scaling and show how the initial Gaussian source scale governs the covariance of intermediate probability-path marginals. We further introduce a Representation-Aware Multimodal Diffusion Transformer (RA-MMDiT), which jointly updates token-level language and full-resolution motion features through joint attention while adapting temporal information flow to the motion representation: causal attention for incremental features defined by frame-to-frame changes, and bidirectional attention for global features such as absolute joint coordinates. Across different datasets and motion representations, MSFlow achieves state-of-the-art text-to-motion performance. Its global representation variant additionally enables zero-shot, inference-time control over any joint or frame through projection sampling without control-conditioned training, delivering leading motion quality with exact constraint satisfaction.