Flexible human motion created from language and style examples
Flexible Motion Generation from Language and Style References
Computer Vision and Pattern RecognitionGraphicsMachine Learning
Summary
It’s hard to make computer animations that not only follow a story or text but also look the way you want in terms of style, like how fast or expressive the movements are. The authors created FlexMoGen, a system that uses both written descriptions and example video clips to make animations that match the story and the style exactly. Unlike other methods, this system can mix different styles over time and doesn’t need style categories to work. It was trained to handle lots of different combinations and can create smooth, long animations that keep the meaning and the look the user wants.
What this means in practice
- •For game developers: Generate character movements in games that match storylines and desired animation styles using text and example clips.
- •For film animators: Create animated human motions that reflect both script instructions and a director’s chosen movement style for more expressive scenes.
Authors
Kai Weixian Lan, Bodie Criswell, Briana Fedkiw, Zhan Zhang, Joseph Teran, Daniel Holden
Abstract
We introduce FlexMoGen, a novel framework for flexible human motion synthesis conditioned on both natural language descriptions and motion style references. Text prompts are effective at defining semantic content, but they are often limited in capturing fine-grained style details such as timing, limb articulation, and expressive dynamics. A style example clip supplements the text by conveying these nuanced motion characteristics directly, enabling the model to preserve high-level intent while reproducing the desired stylistic traits. Given a text prompt and a style example clip, FlexMoGen generates high-quality motions that preserve semantic content while faithfully reflecting the target style, offering users greater control over the animation generation process. Unlike prior methods that rely on discrete style labels and do not generalize to long or multi-style generation, FlexMoGen learns a variational style encoder without style supervision and supports long, time-varying, multi-style synthesis. Our framework jointly pre-trains the style encoder and a text-to-motion latent diffusion model within a unified architecture, modulating motion style through a lightweight adaptation module. It integrates an efficient relative positional encoding scheme and is trained on both stylized and non-stylized datasets, enabling strong generalization to unseen text-style combinations. Experiments show that FlexMoGen achieves the best balance between content fidelity and style reflection.