High-fidelity 3D human scans enable better avatar images
Revisiting Avatar-As-Image: High-Fidelity Registration is All You Need
Computer Vision and Pattern Recognition
Summary
To create realistic digital avatars of clothed people, 3D body scans need to be precisely aligned to a standard body model. The authors show previous methods failed to do this well, limiting the quality of avatar images. They introduce AvaImg, a new technique that sharply improves this precise alignment and adds fine surface details. Their method produces avatar textures so close to real scans that they can be efficiently processed by existing 2D image models.
What this means in practice
- •For game developers: Create highly detailed and accurately textured 3D avatars from real-world scans for immersive characters in games.$Commercial implications: Enables production-quality avatar creation tools that improve realism for game characters, reducing manual modeling costs.
- •For virtual try-on developers: Generate precise 3D clothed human models to improve virtual clothing fitting and visualization for online retail.$Commercial implications: Supports realistic and scalable virtual fitting rooms by providing accurate body and clothing alignment from scans.
Authors
Margaret Kostyrko, Yuxuan Xue, Garvita Tiwari, Gerard Pons-Moll
Abstract
The representation of 3D clothed humans as standardized 2D UV texture and displacement maps over an underlying body model has long been studied. This compact representation is enticing as it enables pretrained image networks to process, generate, and edit 3D avatars, but is only useful if scans are accurately aligned and brought into correspondence via high-fidelity registration. This prerequisite has never been met, which we argue explains the limited quality of prior UV-based methods for clothed humans. Despite its significance, no public method produces high-fidelity SMPL(-X)+D registrations with UV texture from arbitrary clothed scans. We present AvaImg, a multi-stage optimization pipeline, to close this gap: it enforces body-inside-clothing constraint via signed winding numbers, made viable by a three-level efficiency cascade (~10x runtime reduced, ~95% storage saved), and recovers fine surface detail using coarse-to-fine displacement optimization. AvaImg outperforms all baselines in body fitting, shape estimation, and surface registration across six datasets, yielding textured registrations near-indistinguishable from scans (PSNR=34.48dB). For validation of AvaImg's Avatar-as-Image representation as imminently compatible with image foundation models, we auto-encode our UV maps via the frozen FLUX VAE. This achieves only 0.76mm added Chamfer error relative to scan and shows that the resulting maps lie within natural-image distributions, supporting the use of 2D generative priors for 3D avatar generation. Code, data, and Singularity containers will be at https://yuxuan-xue.com/avaimg.