Closed-form method improves uncertainty estimates for 3D face models
Modeling Uncertainty Propagation in 3D Morphable Models
Computer Vision and Pattern RecognitionMachine Learning
Summary
When creating 3D models of human faces, it’s important to understand how certain or uncertain the model’s details are. The authors studied how to quickly and accurately turn uncertainties in the model’s settings into uncertainties for each point on the 3D face. Their new method, called CLUE, does this without heavy computation and helps build better models that can also tell us how confident they are in their predictions. Using CLUE during training and testing makes 3D face reconstructions more accurate and provides useful uncertainty maps.
What this means in practice
- •For computer vision engineers: Improve training of 3D face reconstruction systems by integrating fast, accurate uncertainty estimation for better prediction reliability.
- •For animation and game developers: Create more realistic and interpretable 3D character faces by using detailed uncertainty maps to guide mesh adjustments and animations.
Authors
Megh Shukla, Edo Collins, Prashanth Chandran, Karel Petránek, Sergio Orts-Escolano, Mathieu Salzmann, Alexandre Alahi
Abstract
We study the problem of mapping parameter-space variance of a 3D Morphable Model (3DMM) to per-vertex variance. Traditionally, per-vertex variance has been used to visually interpret the learnt parameter uncertainty. However, mapping parameter-space variance to per-vertex using sampling or error propagation is computationally expensive. We address this limitation with Closed-form Local Uncertainty Estimation (CLUE), which transforms the parameter-space distribution of 3DMMs to per-vertex through a closed-form approximation. CLUE emerges from applying the law of total variance to the 3DMM formulation and is modular, computationally inexpensive, and accurate. Subsequently, we study integrating CLUE for in-the-wild human mesh recovery training and inference. We note that CLUE uses the geometry of the mesh as an inductive bias when predicting the variance, leading to improved accuracy. Through extensive qualitative and quantitative experiments, we show that when used in training, CLUE (1) outpeforms probabilistic and deterministic baselines on metrics evaluating joints, mesh and uncertainty, and (2) has the highest uncertainty-to-error spearman correlation, with applications including data mining and interpretable uncertainty. Finally, we share SAM 3D Body (CLUE) which augments the regressor with high resolution uncertainty maps. We make our code available at https://github.com/vita-epfl/CLUE