Transformer model improves motion retargeting for different skeletons
Retargeting Motions to Diverse Skeletons via Learnable Flattening
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Transferring movement from one skeleton shape to another is tricky, especially when the skeletons look very different. The authors created a special transformer-based model that understands skeletons as graphs and can handle new, unseen skeleton types without extra training. This model learns to represent motion in a way that works well even when skeletons vary in structure. Tests show it makes fewer mistakes and produces more natural-looking movements than other methods, and experts prefer its results.
What this means in practice
- •For character animators: Retarget motions reliably onto new character skeletons without needing paired data or retraining for each skeleton type.
- •For game developers: Create diverse animated characters that share motion styles even when their skeletons differ widely, improving production efficiency.
Authors
Kia-Jüng Yang, Fabian H. Sinz, Paweł A. Pierzchlewicz
Abstract
Cross-structural motion retargeting aims to transfer motion between different skeletal topologies. Despite recent progress, existing state-of-the-art models struggle with reliability in zero-shot settings, i.e. skeletons with different topologies which were unseen during training, and recent Transformer-based attempts have failed to outperform specialized geometric methods. We bridge this gap with a Transformer Autoencoder that learns a topology- and translation-invariant latent space. Our core contribution is a learnable flattening of skeletal graphs that captures both local dependencies and global structure. Unlike the standard transformer architecture, which adds positional information to token content, we integrate graph-based positional encodings multiplicatively, a design choice that follows directly from our flattening formulation. The resulting model handles diverse skeletal topologies within a single unified architecture and trains in a fully unsupervised manner, requiring no paired retargeting data. Ablation studies show, that the graph encodings, multiplicative formulation, and Transformer backbone is critical for the performance. In zero-shot evaluations, our method reduces global joint position error by $43-47\%$ over current benchmarks. A user study ($n = 37$), including expert animators, further ranks our approach highest in motion alignment and physical plausibility ($p < 0.05$). These results demonstrate that our model design is key to making transformer architectures effective for motion retargeting, outperforming existing approaches.