Whole-body human object interactions generated in one step efficiently
MFlowHOI: Whole-Body Human-Object Interaction Generation via Part-Aware MeanFlow
Computer Vision and Pattern Recognition
Summary
Making realistic animations of how people use their whole body and hands to interact with objects is hard and usually takes a lot of computer time. The authors propose a new method called MFlowHOI that can create these animations all at once in a much faster way. They organize the different parts like body, hands, objects, and where they touch into a special space to keep everything coordinated. Their approach reduces the time needed and produces better motion and matching with text descriptions than older methods.
What this means in practice
- •For game developers: Generate complex full-body character interactions with objects quickly to improve game animation workflows.
- •For virtual reality creators: Create realistic whole-body human-object interactions for immersive VR experiences with efficient generation.
Authors
Qiang He, Yonghao Zhang, Yinda Zhang, Chenyu Meng, Liufei Ouyang, Haoxiang Hu, Cangjun Gao, Xiaoming Deng, Cuixia Ma, Hongan Wang
Abstract
Whole-body human-object interaction (HOI) generation requires coordinating body and finger motion, object trajectories, and time-varying contacts. Existing methods typically rely on iterative diffusion or test-time optimization, resulting in high inference costs. Direct one-step generation is highly desired in downstream applications due to efficient inference. However, it is challenging because these HOI components differ substantially in structure and scale. We present MFlowHOI, an efficient one-step framework for whole-body HOI generation. A Contact-Aware Multipart RVQVAE organizes body, hand, object, and contact signals into a structured latent space, with a shared decoder preserving their global coordination. An ST-DiT then models temporal and cross-part dependencies in this space and generates a complete interaction in a single MeanFlow evaluation. Predicted object-local contact correspondences further enable geometric refinement and interaction-preserving transfer to photorealistic 3D Gaussian avatars with different body proportions. Experiments demonstrate that MFlowHOI outperforms existing base generators in motion quality and text alignment while substantially reducing inference time.