Remote-Sensing City Layout Extraction with MLLM
2026-08-17 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors propose a method called Code-as-City that turns top-down images of urban areas into executable code representing city layouts. Instead of just marking objects like roads and buildings on a map, their approach creates a detailed city graph and 3D layout that keeps track of object types and their relationships. They use a large language model along with image data to generate this structured representation, which can be compared to traditional remote-sensing masks and also edited as code. Their system was tested on 100 urban scenes and showed measurable accuracy in matching real layouts.
remote sensingurban layout extractionmultimodal large language modelsemantic segmentationcode generationcity graph3D city modelingintersection-over-uniontop-down imagerysemantic masks
Authors
Zigan Zhou, Kai Li, Yupeng Deng
Abstract
Remote-sensing systems usually describe urban content with detection boxes, semantic masks, or vector boundaries. Such outputs locate classes and support image-plane scoring, yet they do not by themselves constitute an executable layout that retains object identities, typed relations, topology, and regeneration rules. Code-as-City instead casts urban-layout extraction from a single top-down image as constrained code generation with a multimodal large language model (MLLM). An image model first produces an aligned five-class semantic layout prior. Three ordered MLLM passes use the image and this prior to recover roads, land-cover regions and relations, and buildings. Deterministic normalization converts the accumulated records into a city graph and a restricted layout program. Executing the program creates a renderable 3D city layout and an orthographic semantic projection over shared geometry. The projection admits pixel-level comparison with remote-sensing masks, while named objects, relations, and editing operations remain available for synchronized regeneration of both views. Evaluated on the 100 scenes of CityLayout-100, the complete framework obtains 41.1% mean intersection-over-union and 48.3% global intersection-over-union. This result provides quantitative evidence that visual observations can be translated into inspectable, editable city code with coupled planar and 3D outputs.