Structured 3d buildings generated from text with mesh proxies
ProxyBuild: Text-Guided Structured 3D Building Generation with Mesh-Anchored Procedural Proxies
Computer Vision and Pattern RecognitionGraphics
Summary
Creating 3D buildings from text descriptions usually results in models that are hard to edit or separate into parts. The authors present ProxyBuild, a new method that uses a special type of intermediate model called Mesh-Anchored Procedural Proxy (MAPP) to organize building parts clearly. This approach breaks down the task into two steps: predicting these proxies and then turning them into final 3D assets that fit together properly. ProxyBuild performs better than earlier methods by avoiding common problems and producing detailed, editable 3D buildings based on textual input.
What this means in practice
- •For virtual reality developers: Create editable, structurally clear 3D buildings from text to enhance interactive virtual environments.
- •For digital twin engineers: Generate detailed and accurate 3D building models from varied sources for reliable simulation and analysis.
Authors
Xiang Tang, Ruotong Li, Xiaopeng Fan
Abstract
Text-guided 3D building generation holds tremendous application potential, yet existing generative models typically output inseparable single meshes or non-interactive rendered representations. While procedural modeling can generate editable buildings with hierarchical structures, rule authoring is laborious, and even with the aid of large language models (LLMs), it remains challenging to effectively solve procedural rules under geometric constraints. In this paper, we propose ProxyBuild, a novel hybrid framework for structured building generation. We introduce the Mesh-Anchored Procedural Proxy (MAPP) as a novel intermediate representation, which tightly anchors building components onto geometric shells, thereby decoupling the generation task into two phases: proxy prediction and proxy-to-asset instantiation. First, we construct a building dataset with MAPP annotations to train our designed face-edge bigraph encoder. By explicitly modeling the feature interactions of topological elements on heterogeneous mesh graphs, this encoder accurately infers the semantic roles of faces and edges. Subsequently, conditioned on textual styles and attribute parameters parsed by LLMs, we accomplish high-precision asset retrieval and assembly by integrating a spatial placement logic with hard constraints. Extensive experiments show that ProxyBuild not only significantly mitigates common issues in building generation such as over-smoothing, component collisions, and structural corruptions, but also accurately parses semantic-free shells from diverse sources. Outperforming prior baselines across various metrics, our method can robustly generate structurally clear, detail-rich, and post-editable 3D buildings from text, thereby providing a reliable and interactive content foundation for downstream applications such as virtual reality and digital twins.