RayOcc: Occlusion-Aware Ray Occupancy Estimation via Gaussian Mixture Intensity
2026-07-20 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors address the challenge of predicting what objects occupy space in 3D from multiple camera images, which is tough because objects can hide behind each other. Instead of guessing just one distance per camera line, they let multiple possible distances exist at once, better handling hidden objects. Their method, RayOcc, uses math to treat occupancy along a ray as multiple overlapping possibilities, improving prediction accuracy. Tests show their approach works better than other similar methods on a standard dataset.
3D semantic occupancymulti-view imagesdepth ambiguityocclusionray tracingGaussian mixture modelPoisson event formulationnuScenes datasetIoUmIoU
Authors
Junho Kim, Seongwon Lee
Abstract
Camera-only 3D semantic occupancy prediction aims to infer voxel-wise scene semantics from multi-view images, yet remains fundamentally challenging due to depth ambiguity and occlusion. Along a single camera ray, multiple spatially separated surfaces may coexist, making occupancy inherently a multi-label existence problem rather than a single-depth estimation task. However, most existing approaches favor a single dominant depth hypothesis per ray, limiting their ability to model volumetric scenes under complex occlusion. To address this limitation, we introduce RayOcc, an occlusion-aware ray occupancy framework that reformulates ray modeling as multi-label existence prediction. Instead of predicting a categorical depth distribution, RayOcc estimates a non-normalized Gaussian mixture intensity along each ray and converts it into interval-wise occupancy probabilities via a Poisson event formulation, allowing multiple occupied hypotheses to coexist without enforcing mutual competition across depth. The predicted mixture components are interpreted as occupancy hypotheses to initialize sparse 3D Gaussian primitives, which are refined and rasterized for semantic occupancy prediction. Experiments on the nuScenes benchmark show that RayOcc achieves state-of-the-art overall IoU and mIoU among the compared Gaussian-based occupancy methods.