Complexity guided method improves realistic human rendering
Complexity-guided Regularization for Generalizable Human Gaussian Splatting
Computer Vision and Pattern Recognition
Summary
Creating detailed 3D images of people from multiple photographs is tricky because complex shapes and textures can look blurry. The authors found a way to guide the placement of tiny 3D shapes called Gaussians based on how complicated each area is. This helps the computer focus on tricky parts and draw them more clearly. Their method also picks small image areas with lots of details for better training. This approach makes 3D renderings of unseen people look sharper and works quickly enough for real-time use.
What this means in practice
- •For game developers: Create sharper real-time 3D models of unseen human characters from photos for immersive gaming experiences.$Commercial implications: Enables game studios to produce detailed avatars for games using multi-view image input, improving visual fidelity and speed.
- •For virtual reality designers: Build realistic human avatars with fine details that adapt quickly to new input views for virtual environments.
Authors
Gun Ryu, Gi-Mun Um, Won-Sik Cheong, Wonjun Kim
Abstract
Recent studies on generalizable human Gaussian splatting have achieved the notable progress in synthesizing novel views of unseen human subjects from multi-view images. Despite this progress, existing methods still struggle to represent fine-grained details of human subjects, which inherently involve high geometric and textural complexity. This is because previous methods do not impose a constraint on how densely Gaussians are placed, which often leads to blurry rendering results in complex regions. To address this problem, we propose a complexity-guided regularization scheme. The key idea is to leverage a complexity prior computed via the local entropy in geometric and textural characteristics of a clothed human. This effectively prevents Gaussians from being coarsely placed in complex regions, which facilitates the faithful reconstruction of intricate structures. Furthermore, we sample local patches predominantly from complex regions according to this complexity prior, which allows the model to be sufficiently supervised by local details. Experimental results on benchmark datasets demonstrate that our method delivers state-of-the-art performance while maintaining the real-time inference speed in generalizable human Gaussian splatting. The code and model are publicly available at https://github.com/DCVL-3D/CGR-GS_release.