CoSAG: Compact Semantic Anchor Gaussians via Training-Free Rate-Distortion Coding
2026-07-11 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors address the problem of large storage needs when attaching detailed features to millions of 3D components in open-vocabulary scene understanding. They propose a new method called CoSAG that separates building the scene representation from storing it, using a clever way to organize and compress the data. Their approach uses spatially organized anchors and smart prediction to greatly reduce the file size without extra training for each scene. CoSAG achieves much smaller storage sizes while maintaining or improving accuracy compared to previous methods.
open-vocabulary3D scene understandingGaussian SplattingCLIPfeature compressionsemantic fieldentropy codingtransmittance-weighted liftmulti-view denoisingrate-distortion
Authors
Yuang Jia, Jinlong Wang, Junhong Lin, Ruiting Dai, Wei Gao
Abstract
Open-vocabulary 3D scene understanding is commonly achieved by embedding 2D vision-language features such as CLIP into a 3D Gaussian Splatting scene, turning it into a text-queryable semantic field. However, attaching a high-dimensional feature to each of millions of Gaussians inflates a single scene to gigabytes, which makes storage and deployment the real bottleneck of these fields. Existing compact methods each learn and ship a per-scene codec, an autoencoder, a quantized codebook, or a distilled feature field, entangling field construction with field storage and never compressing the per-Gaussian assignment that holds the bulk of the cost. We argue that construction and storage should be decoupled, and that storage is a rate-distortion problem over the per-Gaussian binding to a small anchor table, a structure no prior open-vocabulary method compresses. We present CoSAG, which constructs the field without any per-scene training through a closed-form transmittance-weighted lift, spatially grounded semantic anchors, and multi-view denoising, and stores it with a spatially predictive entropy coder that ships no decoder. Because the anchors are spatially grounded, the binding is predictable and therefore highly compressible. The transmittance-weighted lift and multi-view denoising yield a clean, view-consistent assignment, so the entropy coder spends almost no rate on correcting noise and instead codes only the residual against its spatial prediction. CoSAG reaches sub-megabyte storage while matching or exceeding the state of the art across the 2D-rendered, 3D-selection, and dense-LSeg protocols, reducing field size by 37 to 76x relative to LangSplatV2 at higher accuracy.