Text to motion system generates animations for new skeletons without retraining
TopX: Text-to-Motion for Unseen Skeletons from the Skeleton File Alone
Computer Vision and Pattern Recognition
Summary
Creating realistic motion animations usually requires training on a specific skeleton shape, which limits using new characters. The authors introduce TopX, a system that can generate motion for skeletons it has never seen before, using only the skeleton's structure and basic info. This works without needing extra motion clips or adjusting the model for the new skeleton. Their approach lets the system understand any skeleton's layout and produce smoother and more accurate movements compared to other methods.
What this means in practice
- •For game developers: Generate character motions for new game avatars directly from skeleton files without needing extra motion capture data or retargeting steps.$Commercial implications: Allows game studios to create diverse animated characters efficiently, reducing animation costs and time.
- •For 3d animation artists: Create realistic animations for any rigged character using text descriptions and skeleton info, bypassing manual adjustment or retraining.
Authors
Tengjiao Sun, Pengcheng Fang, Xiaoyu Zhan, Dongjie Fu, Xiaohao Cai, Hansung Kim
Abstract
Text-to-motion models generate for the skeletons they were trained on, while a new character arrives as a skeleton file and nothing else. We study text-to-motion for skeletons that no training clip has used, with no clips of the character, no template, no retargeting and no fine-tuning. We present TopX, a 73M-parameter flow-matching transformer that generates joint rotations and a root trajectory for any kinematic tree from its tree, rest pose and joint names. Motion is normalised by the rest pose and the size of the skeleton, and the rest pose enters as a world-frame descriptor that does not depend on how the rig's author oriented each joint, with rest conventions randomised in training. Attention is factorised over the joints and the frames of a motion and carries a spatial--temporal encoding designed for skeleton graphs of any topology: queries and keys are rotated across joints by a heat-kernel coordinate of the skeleton graph, which is defined for every tree and does not change with the order of its joints or the eigenbasis of its Laplacian, and across frames by a rotary encoding of frame order. To measure what this buys, we build SkelShift: four motion corpora under one codec, 143,334 captioned clips on skeletons of 4 to 142 joints, from which 4,232 clips on 2,043 skeletons are held out by clip, rig, tree class or species and organised along topology, joint count, body plan, species and rig source, and scored by motion quality: Fréchet distances on public kinetic features and on kinematic descriptors, foot skating and ground contact. Against UniMate trained on the same clips, on test rigs without a training clip, TopX lowers the descriptor FID from 5.73 to 4.58 when UniMate's guidance is tuned for the kinetic FID, and the kinetic FID from 1.48 to 0.55 when it is tuned for the descriptor FID, also on motions no training skeleton performs; fewer of its frames sink below the ground.