Memory use in 3D models shrinks by smarter color compression
Only What Was Seen: Observation-Gram Compaction of View-Dependent Appearance in 3D Gaussian Splatting
Computer Vision and Pattern Recognition
Summary
3D models that store colors often keep more information than needed, especially since each part of the model is viewed only from certain angles. The authors came up with a way to measure how changing color details affects the final image quality, based on where cameras see the model. This lets them reduce and compress color information more efficiently, improving image clarity and making models smaller, even without extra training. Their method can be applied within existing compression tools to get better results with the same data.
What this means in practice
- •For graphics software developers: Improve 3D model color compression by using observation-based distortion metrics to save memory while preserving image quality.
- •For game engine creators: Reduce memory footprint of view-dependent 3D assets enabling better performance without losing visual fidelity.
- •For augmented reality hardware makers: Shrink 3D scene data sizes with no training to improve AR device responsiveness and battery life.$Commercial implications: This paper enables compact 3D models suited for AR devices, leading to marketable improvements in hardware efficiency and user experience.
Authors
Krzysztof Pietroszek
Abstract
Most of the memory of a 3D Gaussian Splatting model holds spherical-harmonic colour coefficients, yet each Gaussian is seen only from the narrow cone of directions of the training cameras. We turn this into a distortion metric that other compressors can adopt: a per-Gaussian observation Gram matrix, accumulated from viewing directions and blending weights, is the exact first-order map from coefficient changes to squared image error and needs only the model and the camera poses. Under it, degree reduction becomes a closed-form projection that generalises truncation, degree allocation a Lagrangian rate-distortion problem, and vector quantisation the matrix-weighted Lloyd algorithm, of which Compressed3D's quantiser is the scalar case. Swapped into Compressed3D with everything else unchanged, the metric raises PSNR by +0.49 dB before fine-tuning, with SSIM and LPIPS following, and at matched rate still gains +0.32 dB without a single training image. A training-free stack built on the metric alone is 15% smaller than the image-free GSICO at equal quality on Mip-NeRF 360.