Luminance drives better 3D shape learning than color in Gaussian splatting
What Builds the Scene? Luminance Dominates Geometry Formation in 3D Gaussian Splatting
Computer Vision and Pattern RecognitionGraphics
Summary
Creating 3D scenes from images involves learning both shape and color. This paper shows that the brightness (luminance) of a scene plays a bigger role in accurately forming its 3D shape than the color information alone. The researchers tested this by separating brightness and color during training and found that shapes learned from brightness alone were almost as good as those learned from full color. When color data was removed, the appearance could still be restored well on the fixed shapes formed mostly by brightness information.
What this means in practice
- •For 3d graphics engineers: Improve 3D scene reconstruction methods by focusing model training on luminance signals for more accurate geometry.
- •For digital content creators: Enhance rendering pipelines by separating shape and color learning to better manage scene appearance adjustments post-geometry formation.
Authors
Rezvan Joshaghani, Steven Cutchin
Abstract
Standard 3D Gaussian Splatting (3DGS) learns geometry and appearance jointly from RGB supervision, making it difficult to isolate how luminance and chroma contribute to the learned representation. We study this by training models under different channel supervision, freezing their non-appearance parameters (position, scale, rotation, and opacity), and re-estimating appearance with the same solver before comparing held-out reconstruction. Across eleven benchmark scenes with four independent runs each, geometry learned from luminance alone supports held-out reconstruction 0.085 dB below RGB-trained geometry on average. If chroma is deleted from a trained model, a sufficiently expressive solver can re-fit it on the frozen geometry to the original quality or slightly better. Higher-order spherical harmonics contribute much more reconstruction quality to luminance than to chroma, improving PSNR by 1.44 dB versus 0.19 dB on average, although on mirror-like surfaces hue does still change with viewpoint. The luminance advantage is even larger when geometry is being formed. Chroma-only supervision produces geometry 3.9-5.5 dB worse than luminance-only supervision after the same appearance solve; densification explains part of this gap. Overall, geometry formation in standard 3DGS is strongly luminance-dominated but not luminance-exclusive, and much of the chromatic appearance can be recovered after spatial support has formed.