Animatable and relightable human avatars created from images
ARS-Avatar: Animatable and Relightable Surfel Avatars with Learnable Ambient Occlusion
Graphics
Summary
Creating realistic digital human avatars that can move and respond to light like real people is difficult because movements, materials, and lighting affect how the avatar looks. This paper presents a new way called ARS-Avatar that builds detailed 3D avatars from photos taken from many angles, even without knowing the lighting conditions. The method uses a special 3D representation and learns how different parts of the body should deform and appear under different lighting. Their approach also estimates how shadows and light interact with the avatar, making animations and lighting changes look more realistic.
What this means in practice
- •For game developers: Create high-quality animated and realistically lit human characters from photos without detailed lighting setup.$Commercial implications: Enables realistic avatar creation tools for games, appealing to developers seeking lifelike characters from simple image captures.
- •For virtual reality creators: Generate accurate and animatable avatars that respond naturally to new lighting and poses for immersive virtual experiences.
Authors
Jiateng Liu, Hao Gao, Junxin Sun, Mengqi Liu, Jiu-Cheng Xie, Jucheng Song, Feng Xu
Abstract
Creating animatable and relightable human avatars from multi-view images remains challenging, as pose-dependent deformation, materials, and light visibility are intrinsically coupled in images. In this paper, we present ARS-Avatar, a novel method using surfel representation for high-quality, animatable, and relightable human avatars from multi-view images captured under unknown illumination. We first extract deformation priors from the template mesh and leverage as additional details beyond driving poses to facilitate faithful estimation of surfel attributes and reconstruction of animatable avatar. To support relighting, the deferred shading is employed to estimate BRDF materials. We further introduce a differentiable screen-space ambient occlusion formulation that enables gradient-based optimization of body-part specific occlusion radii through finite differences, providing an efficient approximation of light visibility that can be jointly optimized with the avatar. Extensive experiments demonstrate that ARS-Avatar achieves high-fidelity appearance reconstruction and physically-based material estimation, while enabling realistic animation and relighting under novel poses and illuminations.