StreetDiff improves urban street scene generation with consistent views
StreetDiff: Multi-view Street Scenes Generation via Cross-view Consistent Multi-view Stable Diffusion with Structure Prompts
Computer Vision and Pattern Recognition
Summary
Generating realistic images of city streets from different camera angles is hard because objects can appear duplicated or distorted when views change. The authors created StreetDiff, a new method that keeps these different views aligned by using a special way to connect panoramic and perspective images. They also built a large dataset of urban panoramic images, called Street360, to help train and test their method. This approach produces more coherent and detailed street scene images than previous techniques.
What this means in practice
- •For urban planners: Generate consistent multi-view street scenes to visualize urban design changes more realistically.
- •For game developers: Create realistic, consistent 3D city environments from multiple camera angles for immersive gameplay.$Commercial implications: Improves realistic city scene generation, allowing game studios to offer more immersive virtual worlds.
Authors
Qi Zhang, Yanyifan Wang, Weiyuan Zhang, Hui Huang
Abstract
Multi-view diffusion models have shown strong performance in scenes with strong geometric priors and sparse semantics, such as indoor rooms or simple outdoor environments (e.g., fields, courtyards). However, they often fail to maintain cross-view consistency under camera rotation, especially in structurally complex urban environments. Without explicit modeling of spherical correspondence across views, existing approaches tend to produce object duplication, structural distortion, and layout inconsistency. To address this limitation, we propose StreetDiff, a multi-view diffusion framework that explicitly enforces cross-view alignment during denoising. StreetDiff introduces a Panorama--Perspective Synergy design to decouple global layout reasoning from local detail synthesis, and incorporates a Panorama Alignment Module (PAM) that establishes spherical-projection-based attention constraints across views. By injecting structured alignment constraints without modifying the diffusion backbone, our framework achieves robust cross-view coherence in challenging urban street scene generation tasks. In addition, we construct Street360, a large-scale HDR multi-view urban panorama dataset. Extensive experiments demonstrate that StreetDiff significantly improves structural consistency and visual fidelity compared to prior multi-view diffusion generation methods.