ZARA builds animatable 3D human avatars with realistic lighting from one image
ZARA: Zero-shot Animatable and Relightable 3D Avatar reconstruction from a single image
Computer Vision and Pattern Recognition
Summary
Making 3D models of people that can move and have realistic lighting from a single picture is hard because lighting and the skin’s real colors get mixed up. The authors created ZARA, a method that builds 3D avatars with separate materials and lighting details quickly without needing to customize or fine-tune for each person. It uses a smart combination of a big human reconstruction model and a material-learning system to keep fine details and color accurate. This way, the avatars can be animated and relit realistically in different environments.
What this means in practice
- •For game developers: Generate animatable and relightable 3D human characters from a single photo without person-specific tuning for faster game asset creation.$Commercial implications: Enables selling customized 3D avatars with editable materials and lighting for games based on user photos.
- •For visual effects teams: Create realistic 3D human models with separate material properties from a single image for virtual scene lighting and effects integration.
Authors
Yixiao Chen, Dongdong Weng, Chengfeng Zhao, Tao Gong, Zhihe Zhao, Zhe Zhu, Yue Liu, Yuan Liu
Abstract
Recent large human reconstruction models create remarkable animatable 3D avatars from a single image, but their appearance remains entangled with the illumination and cannot be relighted. In contrast, existing 3D relightable-avatar reconstruction methods typically require person-specific optimization. We present ZARA, a zero-shot feed-forward framework that reconstructs an animatable 3D Gaussian avatar together with its intrinsic attributes from a single image. ZARA couples a pretrained large-scale human reconstruction model with a Material Transformer to inherit geometry priors while learning material-aware Gaussian representations. To preserve localized details, we introduce Gaussian-aligned material feature fusion strategy, which injects multi-scale material features onto individual Gaussians and combines them with geometry tokens. As most human-avatar datasets lack ground-truth material maps, we use pseudo-material labels as conditioning inputs and supervision targets during training. We further introduce MatFace regularization to preserve facial structure and color in albedo prediction under unreliable facial pseudo-material supervision. Extensive experiments demonstrate that ZARA reconstructs animatable and relightable humans within tens of seconds without person-specific optimization or post-processing, outperforming existing baselines in both relighting quality and materials generalization ability. As the ability of recovering the physically-based-rendering attributes of the human, ZARA also enables direct material editing of the avatar's appearance.