Pano2World: End-to-End 3D Generation via Unified Multi-View Sequences
2026-07-01 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors developed Pano2World, a method that turns a single indoor panoramic photo into a 3D scene you can explore from different viewpoints. Unlike earlier approaches that either add errors step-by-step or are limited by video-like constraints, this method creates a 3D Gaussian model and refines it using a special attention technique to keep all views consistent. They also introduced a new way to preserve important hidden details during processing, improving the final 3D reconstruction. Their experiments show this method works better than past ones at generating new panoramic views from different positions.
panoramic image3D Gaussian scenenovel-view synthesisdiffusion modelView-Aware Attention Routinglatent featurescene reconstructioninpaintingindoor scenegeometry
Authors
Zhenjia Li, Jinrang Jia, Yifeng Shi
Abstract
A single panorama captures the full visual sphere from one camera center, yet confines users to looking around in place without enabling true scene exploration. Converting a single panorama into a persistent, renderable 3D representation for free-viewpoint navigation has attracted growing interest; existing methods either adopt iterative per-view completion that propagates inpainting results to update the underlying geometry, leading to progressive error accumulation and cumbersome multi-step pipelines, or leverage the temporal consistency priors of video generation models, yet the continuous-trajectory constraint intrinsic to such models limits their flexibility in covering scenes from multiple directions simultaneously. We present Pano2World, which takes a single indoor panorama as input and directly outputs a persistent, explorable 3D Gaussian scene. Given the source panorama, Pano2World first reconstructs a coarse 3D Gaussian proxy and renders it at adaptively sampled nearby poses to obtain geometrically aligned guidance panoramas; a panoramic diffusion model then jointly denoises all target views via View-Aware Attention Routing, where each target view simultaneously receives geometric constraints from its corresponding guidance panorama and global semantic guidance from the source panorama, naturally enforcing cross-view consistency. To avoid the information loss incurred by decoding the multi-view hidden features formed during joint denoising back to the pixel domain via VAE, we introduce Latent Feature Adapter, a geometry-aware bridge module that directly distills these hidden features into a scene latent, subsequently decoded into the final 3D Gaussian scene. Experiments demonstrate that Pano2World significantly outperforms existing methods on the multi-position panoramic novel-view synthesis benchmark.