DrawingVQA: A Real-World Benchmark for Multi-Depth Visual-Textual Reasoning on Construction Drawings
Artificial IntelligenceComputer Vision and Pattern Recognition
Summary
The authors created DrawingVQA, a test designed to see how well multimodal large language models (MLLMs) understand real construction drawings, which combine images, symbols, tables, and technical text. These drawings are more complex than regular pictures or simple floor plans because they mix visual and textual information used in engineering work. They collected 33 real drawings and 92 questions that require different levels of thinking, from basic observation to expert reasoning. When testing current AI models, the authors found that these models still struggle, especially with more complex questions. This work helps guide future AI improvements for understanding specialized engineering documents.
Authors
Yoonhwa Jung, Junryu Fu, Mani Golparvar-Fard
Abstract
We introduce DrawingVQA, the first benchmark designed to evaluate multimodal large language models (MLLMs) on real-world construction drawings -- a core media in architecture, civil, and many other engineering practices. Unlike natural images or schematic floor plans, construction drawings fuse abstract geometry, symbolic notation, tabular data, annotations, and domain-specific text, forming a uniquely complex visual-textual domain core to engineering workflows. DrawingVQA bridges this gap with 33 "Issued for Construction" drawings and 92 expertly curated question-answer pairs, spanning three reasoning depths: perceptual understanding, contextual interpretation, and domain-expert reasoning. To evaluate model capabilities, we present a dual categorization framework to jointly analyze performance across seven construction-engineering and four MLLM capability dimensions -- the first to explicitly map engineering workflows to AI reasoning competencies. Evaluations of state-of-the-art MLLMs reveal a substantial gap between model and expert performance, particularly at higher reasoning depths. This benchmark lays a foundation for domain-specialized multimodal reasoning to allow for advancement on integration of AI-driven understanding and real-world engineering workflows.