Refining 3d human body shapes without mesh correspondences
Mesh-Agnostic Refinement Models for Latent Space Encoding of 3D Human Body Shapes
Computer Vision and Pattern Recognition
Summary
3D models that represent human bodies often have small errors due to limits in their design or how they are made. The authors propose a new way to improve these models by comparing the 3D shape with a target shape, even when the two models don’t line up exactly point for point. They introduce three techniques: one adjusts the hidden code behind the model, another moves points on the model using a special neural network, and the last one directly tweaks the surface like bending elastic material. This helps make 3D human body models more accurate and flexible.
What this means in practice
- •For 3d animation artists: Improve the accuracy of character models by refining body shapes without needing exact mesh matching.
- •For virtual try-on technology developers: Enhance virtual clothing fit by better aligning 3D human body shapes with varied input scans without mesh correspondences.$Commercial implications: Enables improved virtual fitting solutions by refining body shape models for fashion retailers, enhancing consumer experience.
Authors
Chaima Zouaouid, Jonathan Cerqueirab, Sylvain Arguillere, Martin Bauer, Nicolas Charon, Mohamed Daoudi, Emmanuel Hartman
Abstract
Parametric human body models such as the Skinned Multi-Person Linear (SMPL) model provide compact representations of human shape and pose, but their reconstructions remain affected by both errors in estimated latent parameters and limitations in the decoder's expressive capacity. We propose a framework for refining these reconstructions by exploiting the geometric discrepancy between an initial decoded surface and a target mesh, without requiring vertex correspondences between them. The framework comprises three complementary approaches. Latent-VaRefine uses varifold-distance gradients to correct the estimated latent code while preserving the original parametric representation. Vertex-VaRefine processes multiscale varifold-gradient features with a DiffusionNet backbone to predict vertex displacements in a canonical pose, yielding an approximately pose-equivariant refinement pipeline. ElaOpt directly optimizes surface deformations under elastic regularization and normal-direction constraints.