Spherical pixel method cuts VR image decoding time and size
HEALPix-Based Implicit Neural Compression of Omnidirectional Images
Computer Vision and Pattern Recognition
Summary
Decoding high-quality 360° images quickly is important for virtual reality to feel smooth. The authors propose a new way to compress these spherical images that avoids distortions from common flat map methods. They use a special spherical pixel system called HEALPix to represent and compress images more efficiently. This approach reduces the file size and speeds up decoding compared to state-of-the-art methods, making VR experiences more responsive and visually accurate.
What this means in practice
- •For vr platform developers: Reduce storage and speed up playback of 360° images in VR systems with a new spherical compression technique.
- •For video streaming engineers: Implement efficient decoding of spherical video frames to lower bandwidth and improve user experience for live VR broadcasts.
Authors
Yulong Cheng, Youneng Bao, Junfeng Zhou, Mu Li, Jie Wen
Abstract
Real-time decoding of omnidirectional images is essential for immersive virtual reality (VR) experiences. Compared with conventional transform-based learned image compression (LIC), overfitted codecs offer lower decoding complexity by representing each image with a lightweight implicit neural representation (INR). However, existing approaches typically operate on the equirectangular projection (ERP), which introduces nonuniform spherical sampling and spatial distortion, leading to redundant sampling near the poles and latitude-dependent receptive fields. To address these limitations, we propose a spherical overfitted codec based on a multi-scale Hierarchical Equal Area isoLatitude Pixelization (HEALPix) representation, which enables equal-area sampling and efficient hierarchical modeling of spherical signals. Building on this representation, we develop a Spherical Inter-Scale Conditional entropy model that exploits spatial dependencies and cross-scale correlations among HEALPix latents to improve coding efficiency. We further introduce a Multi-Ring Spherical Convolution that aggregates neighboring pixels over a nearly uniform spherical receptive field across latitudes. Experiments on a public omnidirectional image dataset demonstrate BD-rate reductions of 16.3%, 26.5%, and 19.5% over VVC in terms of V-PSNR, V-MS-SSIM, and V-LPIPS, respectively. Our method also achieves a 13.1% BD-rate reduction over ERP-based Cool-Chic 5.0 in terms of V-PSNR, while providing 2.6-3.6x faster decoding than the evaluated learned autoencoder codecs.