Model improves motion transfer between characters with different skeletons
ReFM: Semantic-Aware Refinement Flow Model for Motion Retargeting
Computer Vision and Pattern RecognitionArtificial IntelligenceGraphics
Summary
Motion retargeting means copying actions from one character to another with different body shapes, but it's hard to keep the action meaning and realistic movement. The authors tackle two problems: learning the important action features without perfect matching pairs, and deciding how to retarget when no exact correct answer exists. They created ReFM, a model that adjusts the copied motion step-by-step to keep the meaning, look natural, and flow smoothly over time, without relying on any specific character's shape. This method works with different starting techniques and improves how motions fit new characters.
What this means in practice
- •For game developers: Generate character animations that adapt naturally to different skeletons while preserving intended actions and smooth movement.
- •For visual effects artists: Refine motion capture data for diverse characters to maintain semantic meaning and physical realism in animation pipelines.
Authors
Jingxiang Qu, Lucie Taglienti, Evan Atherton
Abstract
Motion retargeting transfers motion across characters with different skeletal structures while preserving semantic intent and physical plausibility. Despite recent progress, two fundamental questions remain: (i) how can reliable source-motion semantics be learned without high-quality paired retargeting data, and (ii) how should retargeting be formulated when no reliable paired motion can serve as a definitive regression objective? Existing methods commonly preserve semantics by constraining predictions toward copied motions. However, such initializations entangle useful articulation cues with artifacts caused by mismatched skeletal proportions and body geometry. Moreover, directly regressing a final motion in one forward pass is restrictive because retargeting is inherently underdetermined, and the desired solution must balance semantic fidelity with target-specific physical and temporal constraints rather than match a unique paired target. Motivated by these limitations, we propose ReFM, a source-mesh-agnostic, energy-guided model that reformulates motion retargeting as progressive refinement. First, an SO(3) canonicalizer removes redundant global-orientation variations. Second, a cross-character semantic encoder, pretrained through contrastive learning, provides a character-invariant representation for both optimization guidance and semantic evaluation. ReFM then progressively refines an initialized target motion through a learned flow guided by semantic consistency, physical plausibility, temporal coherence, and minimal motion modification. The framework is compatible with different initialization strategies, including both direct motion copying and Autodesk HumanIK, an industry-standard full-body inverse-kinematics retargeting system.