MaterialClusterGS: Palette-Based Material Decomposition and Physically-Based Relighting with 2D Gaussian Splatting
2026-06-08 • Graphics
Graphics
AI summaryⓘ
The authors created MaterialClusterGS, a method that breaks down materials in 3D scenes into a small set of shared material types to make editing and lighting easier. Instead of treating each tiny piece of the scene as having its own separate material, they group similar materials together using a global palette. This helps avoid confusion caused by shadows and lighting effects that can mess up material guesses. Their approach also models how light interacts with surfaces more realistically and allows changes to one material to apply consistently across the scene. This makes tasks like changing colors or lighting more practical and reliable.
2D Gaussian SplattingBRDFmaterial decompositioninverse renderingphysically based renderingpalette-based appearancerelightingmaterial editingenvironment lightingspatial material field
Authors
Hao Zhang, Ang Li, Boyan Du, Junke Zhu, Fei Zhu, Meng Gai, Zhangjin Huang, Guoping Wang, Sheng Li
Abstract
We present MaterialClusterGS, a palette-based material decomposition framework for 2D Gaussian Splatting that enables physically based relighting and material editing. Existing Gaussian inverse rendering methods typically assign independent BRDF parameters to individual primitives. While flexible, this local fitting strategy makes material recovery highly under-constrained: shadows, indirect illumination, geometric errors, and visibility residuals can be absorbed into thousands of slightly different local material estimates. Meanwhile, recent palette-based appearance methods operate solely in RGB space without modeling physical materials or illumination. To bridge this gap, we represent scene materials using a compact global palette of shared BRDF prototypes assigned via a continuous spatial material field. Without shared material structure, editing one region does not propagate consistently to others of the same material, making per-primitive decompositions impractical for editing. We jointly optimize the material field, palette prototypes, and environment lighting under a physically based rendering objective. The resulting framework recovers compact, spatially coherent attributes directly usable for material editing, relighting, and transfer.