Omnidirectional image compression boosts virtual reality rendering speed and quality
Rate-Distortion Adaptive Primitive Selection for Omnidirectional Gaussian Splatting
Computer Vision and Pattern Recognition
Summary
Virtual reality needs very fast and good-quality images so users feel immersed, but current methods often force a trade-off between speed and image quality. The authors created a new way to represent these images using Gaussian dots arranged on a special sphere grid, which makes encoding more efficient and lets the system focus on the part of the image the user is looking at. Their method adapts by removing less useful dots automatically to keep the data size small, all while supporting smooth and fast loading of images. This approach shows better image quality and much faster rendering compared to previous methods in tests.
What this means in practice
- •For virtual reality developers: Deliver high-quality 360-degree video and images with reduced latency by efficiently encoding data focused on the user’s current view.
- •For graphics engine developers: Implement faster rendering pipelines by using hierarchical spherical grids and adaptive primitive removal for immersive environments.
Authors
Yulong Cheng, Youneng Bao, Junfeng Zhou, Mu Li, Jie Wen
Abstract
Learned image codecs (LICs) achieve high reconstruction quality, but their decoding speed is often insufficient for immersive virtual reality (VR). Gaussian splatting (GS) codecs render much faster, yet still lag in reconstruction quality and typically decide primitive allocation without considering the coding cost of each primitive. We introduce OIC-GS, an omnidirectional GS codec with a new hierarchical HEALPix primitive grid representation. Gaussian primitives are anchored at predefined spherical locations, eliminating explicit coordinate coding. Finer levels refine their coarser ancestors, naturally supporting coarse-to-fine reconstruction and layered transmission. The predefined grid also enables efficient viewport decoding by selecting only view-relevant primitives. We further introduce a lightweight entropy model for quantized primitives and optimize the codec under a spherical rate-distortion objective. Primitives with insufficient rate-distortion benefit are automatically removed when their quantized opacity becomes zero, allowing OIC-GS to adapt both primitive density and level of detail without a fixed primitive budget. A single bitstream supports full-sphere, viewport-dependent, and progressive decoding. The first viewport reaches final quality after decoding only 52% of the bitstream, and is then rendered at 1,270 FPS. On a 100-image omnidirectional benchmark, OIC-GS outperforms all evaluated GS codecs, reducing WS-PSNR BD-rate by 49.6% over GaussianImage++ and 68.6% over SGI, which uses a learned entropy model.