Generative 3D home models created from few panoramic photos
Holi-Spatial 360: Generative Reconstruction from Sparse Panoramas for Spatial Rendering and Intelligence
Computer Vision and Pattern Recognition
Summary
Capturing full 3D models of multi-room indoor spaces usually needs lots of expensive hardware and detailed scans. The authors developed a system that builds detailed and realistic 3D models using just a few panorama photos taken in homes, which are easy to get from real estate sites. Their method cleverly combines different computer vision techniques to fill in missing parts and check accuracy, producing large datasets with room layouts and object labels. They also tested how well visual AI understands these reconstructed spaces, finding improved results when trained on their dataset.
What this means in practice
- •For real estate platform developers: Create detailed 3D home models with labeled rooms and objects from a few panoramic photos to enhance virtual tours and property presentations.$Commercial implications: Provides real estate platforms with scalable, photorealistic 3D home visualizations enhancing user engagement and sales.
- •For augmented reality app developers: Generate coherent multi-room 3D scenes from sparse panoramas to enable AR navigation and interaction inside real homes without extensive scanning.
Authors
Yuanyuan Gao, Zhirui Liu, Yuning Gong, Yifei Liu, Yifan Zhan, Kuixuan Jiao, Xiaolong Zhou, Yifan Yang, Hongjie Zhang, Xue Yang, Dan Xu, Zhihang Zhong
Abstract
Spatial intelligence requires scalable 3D environments that are both photorealistic and semantically grounded, yet existing indoor datasets are dominated by room-scale captures and costly LiDAR acquisition. Online real-estate platforms offer abundant panoramas of diverse multi-room homes, but sparse viewpoints and texture-poor regions make reconstruction severely under-constrained. We introduce Holi-Spatial 360, a generative reconstruction and data curation framework for converting these panoramas into annotated multi-room 3D environments. Because reconstruction quality bounds subsequent generation, our Correspondence-Guided Geometric Optimization converts feed-forward reconstruction priors into reliable cross-view constraints and builds a coherent 3D Gaussian Splatting representation as 3D spatial memory. Adapted single-image and video priors then restore appearance and complete continuous trajectories while preserving observed content. The reconstructed scenes support arbitrary RGB-D rendering and an automated curation pipeline that combines end-to-end 3D grounding with VLM-based verification. Holi360 contains over 1K multi-room 3DGS scenes, more than 100K instance annotations, and 2M spatial question-answer pairs. We also introduce the Holi360 benchmark to evaluate the spatial understanding of VLMs in real-world, multi-room indoor environments. Experiments demonstrate superior reconstruction and curation quality under sparse panoramic inputs, while fine-tuning VLMs on Holi360 substantially improves spatial reasoning capabilities.