Astra agent builds detailed 3D building models without training
AstraLOD3: Zero-shot multimodal agentic reconstruction of LOD3 building models
Computer Vision and Pattern Recognition
Summary
Creating detailed 3D building models often requires specially designed tools or training for each building type, which can be complicated and less flexible. The authors studied whether a general AI model called Astra can build these detailed models automatically, using images, camera data, and sparse 3D points, guided by natural language instructions. They showed Astra can reconstruct high-quality 3D models nearly as well as specialized methods without prior training on the task. Their approach lets the AI choose and run different procedures dynamically, pointing toward more flexible and adaptable building reconstruction tools in the future.
What this means in practice
- •For urban planning teams: Automatically generate detailed 3D building models from mixed image and point cloud data without custom training for each city area.
- •For videogame environment artists: Construct accurate and detailed building models from diverse photos and scans without manual modeling or dedicated pipelines.
Authors
Bryan G. Pantoja-Rosero
Abstract
Automated LOD3 building modeling typically relies on purpose-built geometric or learning-based pipelines, limiting flexibility across heterogeneous buildings and input evidence conditions. This study investigates whether Astra, a general-purpose multimodal foundation model, can address these limitations through zero-shot reconstruction of LOD3 building models within an agentic framework under bounded autonomy. AstraLOD3 combines multi-view images, calibrated cameras, and a filtered sparse SfM point cloud with a natural-language reconstruction specification, while the Astra agent dynamically selects and executes computational procedures using Python and Blender. Across 35 runs, including 24 benchmark buildings, AstraLOD3 achieved a mean FRDS of 0.9647 and geometric agreement comparable to that of previous purpose-built methods. Controlled ablations further revealed the effects of reconstruction guidance, evidence modalities, model configuration, and run-to-run variability. The results demonstrate that structured LOD3 reconstruction can be formulated as a constrained agentic process rather than as a fixed pipeline. Future work will investigate adaptive refinement, user-guided correction, task-specific specialization, and damage-aware reconstruction.