Flexmogen creates human motion from text and style examples
Flexible Motion Generation from Language and Style References
Computer Vision and Pattern RecognitionGraphicsMachine Learning
Summary
Creating realistic animations of human movement from text alone can miss important style details like timing and expression. The researchers developed FlexMoGen, a system that uses both text descriptions and example motion clips to generate animated movements that capture both the meaning and style. This approach does not need predefined style categories and can combine multiple styles over time. The system is trained on diverse data and can produce high-quality, flexible animations that closely match the input text and style example.
human motion synthesisnatural language processingmotion stylelatent diffusion modelvariational style encoderstyle transferanimation generationpositional encodingmulti-style synthesis
Authors
Kai Weixian Lan, Bodie Criswell, Briana Fedkiw, Zhan Zhang, Joseph Teran, Daniel Holden
Abstract
We introduce FlexMoGen, a novel framework for flexible human motion synthesis conditioned on both natural language descriptions and motion style references. Text prompts are effective at defining semantic content, but they are often limited in capturing fine-grained style details such as timing, limb articulation, and expressive dynamics. A style example clip supplements the text by conveying these nuanced motion characteristics directly, enabling the model to preserve high-level intent while reproducing the desired stylistic traits. Given a text prompt and a style example clip, FlexMoGen generates high-quality motions that preserve semantic content while faithfully reflecting the target style, offering users greater control over the animation generation process. Unlike prior methods that rely on discrete style labels and do not generalize to long or multi-style generation, FlexMoGen learns a variational style encoder without style supervision and supports long, time-varying, multi-style synthesis. Our framework jointly pre-trains the style encoder and a text-to-motion latent diffusion model within a unified architecture, modulating motion style through a lightweight adaptation module. It integrates an efficient relative positional encoding scheme and is trained on both stylized and non-stylized datasets, enabling strong generalization to unseen text-style combinations. Experiments show that FlexMoGen achieves the best balance between content fidelity and style reflection.