Point-based method improves facial animation transfer without artifacts
PDB: Point-Based Deformation Blending for Facial Animation Retargeting
Computer Vision and Pattern RecognitionGraphics
Summary
Transferring facial expressions between different 3D face models is tricky because the models have different shapes, which can cause weird surface glitches. The authors developed a new way called PDB that uses a set of special points and weights to move the face parts smoothly without those glitches. Their method only needs to learn once per target face and works well even when transferring expressions between different people. Tests showed it keeps expressions accurate, looks better visually, and runs quickly compared to older methods.
What this means in practice
- •For game developers: Create realistic facial animations that transfer smoothly between character models with different face shapes, reducing visual glitches.
- •For animation studios: Speed up facial animation workflows by reusing weights for different characters and improving visual quality without manual correction of mesh artifacts.
- •For virtual reality content creators: Enable real-time, high-fidelity facial expression transfers between avatars with diverse face meshes, improving immersion.$Commercial implications: This method enables commercial VR products to offer fast, artifact-free facial expression retargeting that enhances user experience.
Authors
Sihun Cha, Hyeonseung Shin, Suah Yu, Junyong Noh
Abstract
Mesh-agnostic facial animation retargeting transfers expressions across meshes with different structures, but preserving facial motion without surface artifacts remains challenging. To address this, we present PDB, Point-Based Deformation Blending for facial animation retargeting. PDB predicts a compact set of deformed control points from a source neutral-expression pair and blending weights from the target neutral mesh. The weights are computed once per target and reused across frames, while the control points vary with each source expression. ReLU enforces non-negative weights and permits exact zeros, followed by row-wise normalization. The target mesh is reconstructed directly by multiplying the weights and control points, without a predefined cage, precomputed coordinates, a learned per-element deformation decoder, or a global reconstruction solve. Trained only with self-retargeting reconstruction supervision, PDB supports cross-identity transfer without paired cross-identity training expressions. Experiments demonstrate accurate retargeting, fast inference, and localized support in the learned weights. Joint evaluation of expression accuracy and local surface preservation shows reduced surface artifacts relative to the evaluated dense displacement method while retaining the intended motion. Perceptual evaluations further support expression fidelity and visual quality in both self- and cross-retargeting.