Gaussian splatting method creates relightable 3D assets for realistic rendering
GS-PI: An Optimization-Decoupled Appearance Decomposition Approach for Generating PBR Gaussian Assets
Computer Vision and Pattern RecognitionGraphics
Summary
Creating 3D models that look realistic under different lighting is difficult because lighting and material details often get mixed together in automatic methods. The authors developed a new way to separate these elements using a process that works directly on 3D points instead of flat images. This approach keeps details consistent from different viewpoints and prevents shiny reflections from being mistaken as material color. Their method produces 3D assets that can be lit in realistic ways without needing complicated step-by-step adjustments.
What this means in practice
- •For game developers: Generate fully relightable 3D assets from view-based captures for use in physically based rendering game engines.$Commercial implications: Enables game studios to produce realistic, relightable Gaussian-based assets without manual tuning or proxy meshes, improving asset production workflows.
- •For visual effects artists: Create 3D models that maintain consistent appearance under changing illumination for seamless integration into visual effects shots.
Authors
Jieting Xu, Rengan Xie, Zijian Huang, Zehui Jin, Rui Wang, Yuchi Huo
Abstract
Gaussian Splatting (GS) excels at novel-view synthesis but encodes baked-in radiance, tightly entangling illumination with geometry and preventing seamless integration into physically based rendering (PBR) pipelines. Existing inverse-rendering methods attempt to disentangle materials via joint optimization, but often suffer from competing objectives that cause severe ambiguities and residual lighting artifacts. To overcome this, we present GS-PI, a novel optimization-decoupled framework that casts PBR material generation as a geometry-conditioned diffusion process on 3D point clouds. By operating directly in the 3D domain, our method inherently guarantees multi-view consistency, sidestepping the severe pixel correspondence issues that challenge 2D diffusion approaches. We introduce a multi-scale cross-view conditioning mechanism that integrates three complementary components: a global semantic prior, source-anchored photometric cues, and an absolute spatial learned view-direction conditioning signal. This design efficiently compresses complex multi-view evidence, mitigating cross-view projection misalignment and successfully preventing specular highlights from baking into intrinsic colors. By extracting a point cloud from a pre-trained Gaussian model, predicting PBR attributes via conditional diffusion, and distilling them back through differentiable rasterisation, we yield a fully relightable PBR-GS asset. GS-PI outperforms recent inverse-rendering baselines while replacing per-scene joint illumination/BRDF optimization with a learned diffusion pass followed by a short target-driven distillation, without requiring proxy meshes.