UniMate creates motion for any 3D skeleton using text prompts
UniMate: One Unified Model to Animate Diverse Skeletons
Computer Vision and Pattern RecognitionGraphicsMachine Learning
Summary
Animating 3D skeletons usually requires specialized models or extra setup for each type of skeleton. The authors developed UniMate, a single system that can generate movements for many different skeleton shapes just from a 3D asset and a text description. They designed a special transformer model that understands the skeleton’s structure using new techniques for representing joint connections. Training on a large collection of diverse motion data, UniMate works well across many skeleton types without extra tweaking. It can even create new motions or edit animations guided by text.
3D animationriggingskeleton topologytransformerdiffusion modelposition embeddinggraph Laplacianmotion synthesiszero-shot generalizationtext-guided editing
Authors
Linzhan Mou, Jiahui Lei, Zhiyang Dou, Chenyue Cai, Chaoyue Song, Adam Finkelstein, Szymon Rusinkiewicz
Abstract
Recent advances in automatic rigging now deliver animation-ready 3D assets at scale, yet generating the motion to drive them remains a bottleneck. Existing learned animators are topology-constrained: they rely on category-specific templates or require per-skeleton fine-tuning and reference motions at inference. We present UniMate, a unified foundation model that synthesizes articulated motion for arbitrary skeletons from a rigged 3D asset and a text prompt, with no test-time optimization or per-skeleton retraining. UniMate introduces a topology-aware diffusion transformer, which integrates skeletal topology into attention via three mechanisms: (1) a graph-aware attention bias from pairwise joint relations and geodesic distances; (2) a spectral rotary position embedding generalizing RoPE to arbitrary kinematic trees via the graph Laplacian; and (3) a global topological conditioner attention-pooled from the rest-pose skeleton. We also curate UniML3D, 13,006 motion sequences spanning bipedal, quadrupedal, avian, marine, insectoid, serpentine, and articulated rigid objects with unified canonicalization and text pairing. Trained on this dataset, UniMate outperforms state-of-the-art baselines in quality, generalization, and efficiency, and supports zero-shot cross-topology transfer, in-betweening, expansion, and text-guided editing. Our project page is available at https://linzhanmou.com/unimate/.