AI summaryⓘ
The authors address the challenge of making virtual humanoid characters follow high-level text commands realistically using physics. They note that previous methods either struggle with disconnects between planned motions and physical execution or have difficulty linking text directly to low-level actions. Their approach, called MIND, uses a two-level intent system to bridge the gap: one predicts overall behavior, and the other refines actions step-by-step, improving how well the character's movements match the text. By encoding the character's states into a compact form, MIND better understands and generates natural, physically plausible behaviors from text. Experiments show MIND performs better than earlier methods at this task.
Physics-based humanoid controlText-driven behavior synthesisDiffusion modelsBehavioral intentSemantic alignmentKinematic motion generationImitation learningLatent space encodingMulti-scale intent diffusion
Authors
Bin Li, Ruichi Zhang, Han Liang, Jingyan Zhang, Juze Zhang, Xin Chen, Jingya Wang
Abstract
Enabling physics-based humanoids to execute diverse behaviors from high-level textual commands remains a significant challenge. Existing methods typically follow either a two-stage paradigm that combines kinematic motion generation with physics-based tracking, or an end-to-end imitation-learning paradigm that directly generates actions from text. However, the former suffers from the inherent domain shift between kinematic generation and physics-based tracking, while the latter struggles with the substantial modality gap between textual commands and low-level actions, limiting effective semantic alignment. Notably, humanoid states encode rich motion dynamics that are more semantically aligned with textual descriptions than low-level actions, making them a natural basis for deriving behavioral intent. Building upon this insight, we propose MIND, a novel end-to-end diffusion framework for text-driven physics-based humanoid control that leverages behavioral intent as a semantic bridge between textual commands and low-level actions. At its core, MIND introduces a multi-scale intent diffusion mechanism, where a holistic intent predictor captures global behavioral dynamics to guide overall behavior synthesis, while an immediate intent predictor provides step-wise, fine-grained signals for local behavior refinement at each diffusion step. This hierarchical intent formulation imposes a structured inductive bias for humanoid control, improving semantic alignment and behavioral naturalness. Furthermore, MIND encodes humanoid states into a latent space to enable more effective semantic intent modeling. Extensive experiments demonstrate that MIND outperforms existing methods and synthesizes coherent, physically plausible, and semantically aligned humanoid behaviors from text commands. Our code will be released to facilitate future research.