Procedural reconstruction creates editable 3d building models from images
Visual-to-Executable Procedural Reconstruction of Buildings
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
It can be hard to turn pictures of buildings into detailed 3D models you can change or reuse. The authors approach this by first figuring out the main layout of the building’s visible parts, then describing the smaller details in a way that a computer can understand and rebuild. Their method turns pictures taken from normal views into programs that can recreate the building’s facade in 3D. This lets users easily edit or reuse these models later for design or visualization.
What this means in practice
- •For architectural visualization teams: Create editable 3D building facades from ordinary images for use in urban planning and architectural design.
- •For game developers: Generate procedural building models from street images to enrich urban environments with detailed, modifiable assets.$Commercial implications: Enables sale of customizable 3D city assets that developers can adapt and reuse extensively.
Authors
Zichong Zhu, Zhihao Liu, Andrei Sharf, Zhanglin Cheng
Abstract
Procedural modeling offers compact, editable, and reusable representations for architectural content, but building such models still depends heavily on manually authored grammars. We study image-based procedural building reconstruction from in-the-wild images, with the goal of recovering an executable procedural description that can be instantiated as an editable 3D building asset. Because a single image provides only partial and view-dependent evidence, we focus on recovering the dominant visible facade(s) and attach the recovered facade program to a simple mass model for building-level instantiation. Our method decomposes this task into two structured prediction stages. First, a Global Structure Layout Tree recovers facade-scale split-and-repeat organization through a fine-tuned layout model with render-guided visual-symbolic reflection. Second, a Component Parameterization schema maps representative facade tiles to structured procedural attributes for local architectural component synthesis. These two representations are then deterministically compiled into executable Computer Generated Architecture (CGA) programs, avoiding the brittleness of direct low-level program generation. Experiments show that the proposed framework handles both regular facades and challenging in-the-wild images, while maintaining high structural consistency, procedural executability, and fine-grained architectural detail. The recovered programs further enable controllable and reusable 3D building asset generation.