Diffusion models improve 3D city maps from satellite images
Enhancing Photogrammetric Digital Surface Models with Pretrained Diffusion Models and Multimodal Conditioning
Computer Vision and Pattern Recognition
Summary
Satellite images can create 3D maps of cities, but these maps often have mistakes like noise and missing areas. The authors improved these maps by using a special type of AI called diffusion models that learn from both the 3D map data and the original satellite images. They adjusted an existing AI model so it works better for these maps and trained it using high-accuracy LiDAR data. Tests in several French cities showed the improved maps are closer to the true shape of the land.
What this means in practice
- •For urban mapping teams: Produce more accurate 3D city surface models from satellite images, reducing errors without expensive LiDAR scans.
- •For geospatial data providers: Enhance satellite-derived elevation data for better mapping products by integrating imagery and diffusion AI models.$Commercial implications: Enables commercial geospatial services to offer higher-resolution and cleaner 3D surface maps using satellite data with AI enhancements.
Authors
Antoine Lorentz, Stéphane May, Valentine Bellet, Dawa Derksen, Bastien Nespoulous
Abstract
Large-scale Digital Surface Models (DSMs) can be produced cost-effectively from satellite images via stereo-photogrammetry. However, the resulting 3D maps are often contaminated by noise, outliers, and voids. On the other hand, aerial LiDAR provides high-accuracy elevation measurements at a substantially higher cost. In this work, we study diffusion models conditioned both on photogrammetric DSMs and Pléiades imagery to refine vertically co-registered DSMs. We introduce a modified Stable Diffusion 3 architecture with a pruned text stream and a patch-wise normalization strategy, enabling stable training on LiDAR data and transfer from natural images to elevation maps. Experiments in French cities demonstrate that multimodal conditioning improves elevation accuracy, reducing Dense Urban RMSE from 6.00 to 3.45 m in the in-context cities and from 4.16 to 2.77 m in the held-out city of Bordeaux.