Benchmark tests AI and humans on recognizing buildings in multiple image types
ARCH-B: Architectural Representation, Comprehension and Hierarchy Benchmark
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
It can be hard for computers to tell that different pictures, floor plans, or drawings all show the same building. The authors created a new test called ARCH-B with many questions asking which images match the same building across different types. They tried this test on 25 computer models and many people without special training. Some AI models did well, but most struggled, especially linking floor plans to photos, while humans were in the middle in accuracy. This benchmark helps understand where AI models succeed or fail in recognizing buildings from various visual forms.
What this means in practice
- •For architectural software developers: Improve design tools by testing AI’s ability to match floor plans and photos for better building visualization integration.
- •For urban planners: Evaluate AI models that link various building representations to analyze city environments more effectively.
Authors
Kieran Sagar Parikh, Jose Luis Garcia del Castillo y Lopez
Abstract
Multimodal models increasingly interpret visual environments, but their ability to recognize the same building across photographs, floor plans, elevations, sections, and renderings remains poorly characterized. We introduce ARCH-B, a benchmark of 354 four-choice questions across 11 cross-representational archetypes, constructed from a building-linked corpus of 3.9 million architectural images using visually similar distractors, model-guided difficulty screening, and manual validation. We evaluate 25 multimodal models and collect 5,830 responses from non-expert human participants. Model accuracy ranges from 10.45% to 83.90%, compared with a human baseline of 35.35%. Models perform comparatively well on mixed-representation outlier detection and photograph matching, but remain weaker on floorplan-to-photograph correspondence. Human and model difficulty across archetypes is only weakly correlated (Spearman's (ρ=0.33)). Held-out evaluation confirms that the difficulty identified during screening generalizes beyond the curation models. ARCH-B provides a diagnostic evaluation of visual correspondence and representation transfer across architectural media.