Unified system generates 3D motion for humans and animals
UniMo: Unifying Human and Animal Motion Generation
Computer Vision and Pattern RecognitionGraphics
Summary
Generating realistic 3D movements for different animals is hard because animals have different body shapes compared to humans, and data about animal movements is scarce. The authors created a new system called UniMo that represents motion using points instead of skeleton types, making it work for many species. They also made a large new dataset with many motion examples and text descriptions for both humans and animals. Their method performs well across several tests, showing it's possible to generate motions for both humans and animals in one system.
What this means in practice
- •For game developers: Create diverse animated human and animal characters from text descriptions for interactive scenarios without building separate systems per species$Commercial implications: Enables new motion generation tools for gaming studios to produce varied creature animations efficiently from text commands
- •For robotics engineers: Program robots to imitate complex motions of various animals and humans by using a unified model handling different body structures
Authors
Zeyu Zhang, Zhiyuan Zhang, Siheng Wang, Yiran Wang, Danning Li, Ian Reid, Richard Hartley
Abstract
The conditional generation of 3D motion has emerged as a key research topic due to its wide applicability across robotics, AR/VR, gaming, and content creation. However, extending recent advances in text-driven human motion generation to the animal domain remains challenging due to two core limitations. First, animals exhibit highly diverse skeletal topologies, unlike the standard human structure, making unified modeling across species difficult and leading to inefficient per-species models. Second, existing animal motion datasets suffer from limited scale and annotation quality, constraining model performance. To address these challenges, we propose UniMo, a unified point cloud-based motion generation framework that bypasses topological discrepancies by converting parametric skeletons into unparametric representations, further enhanced by dynamic sampling that allocates more points to active joints. Additionally, we present UniML3D, a large-scale motion-language dataset spanning both human and animal categories, containing 145,907 motion sequences and 433,388 captions-over 102x larger than existing animal datasets. Our method achieves state-of-the-art results on UniML3D and three public benchmarks including HumanML3D, KIT-ML, and AnimalML3D, demonstrating the feasibility and effectiveness of unified human-animal motion generation. Website: https://steve-zeyu-zhang.github.io/UniMo.