Learning Spherical Occupancy Profiles for Multi-View 3D Reconstruction and Generation
2026-08-24 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors explore spherical occupancy profiles, which describe how likely parts of a 3D object are occupied along rays from different views, as a shared way to represent 3D shapes from images. They create methods to both predict these profiles from images and generate new 3D shapes using learned models, achieving accurate depth estimates. They also study how to sharpen these profiles to better match true shapes without losing detail. Testing on real photos shows their approach works beyond synthetic data. Overall, the authors show that these occupancy profiles can effectively link 3D reconstruction and generative modeling with uncertainty awareness.
Spherical occupancy profilesMulti-view 3D reconstruction3D Gaussian representationFiLM conditioningVariational Autoencoder (VAE)Latent diffusion modelClassifier-free guidanceDepth estimationShape morphologyDTU dataset
Authors
YiHsuan Tsai
Abstract
We study spherical occupancy profiles-the ray-wise occupancy probability profiles P(r) = T(r) o(r) distilled from multi-view 3D Gaussian reconstructions-as a unified intermediate representation for both discriminative and generative 3D reconstruction from images. On a 999-object subset of Google Scanned Objects with 48 turntable views each, we train (i) a discriminative per-ray decoder that injects global view-averaged and ray-specific image evidence into a FiLM-conditioned profile head, reaching median soft depth error 0.035 (normalized) on an independent 90-object test split, and (ii) a generative pipeline built on a profile VAE and a latent diffusion model, which supports unconditional sampling that matches the reconstruction manifold and image-conditioned multi-solution reconstruction whose per-object solution spread is quantifiable and tunable via classifier-free guidance. We further analyze the morphology of predicted profiles: post-hoc power sharpening and a learned sharpening target both recover ground-truth profile width without degrading depth, exposing a monotonic width-peak frontier in the L1-per-ray loss family and motivating a principled redefinition of morphology gates. Real-photo validation on two DTU scenes confirms the pipeline transfers to non-synthetic input. Our results suggest that ray-wise occupancy profiles offer a compact, learned, and uncertainty-aware interface between multi-view reconstruction and generative priors.