Assessing the Benefits of Combining Advanced Deep Learning Techniques for Post-Disaster Building Damage Assessment from UAV Imagery
2026-08-03 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors address the problem of quickly and accurately assessing building damage after disasters using drone images. They note that traditional computer vision models need lots of labeled data and struggle across different regions, while large vision-language models are good at understanding but not so precise for detecting objects like buildings. To fix this, they combine both: a computer vision model first finds and draws boxes around buildings, then a vision-language model classifies the level of damage and provides context. Their method improved accuracy on two real-world datasets and worked well even with limited labeled data for detection.
Unmanned Aerial Vehicle (UAV) imageryComputer Vision (CV)Large Vision-Language Models (LVLMs)Object detectionDamage assessmentBounding boxFine-tuningRescueNetFloodNetHybrid framework
Authors
Huy Quang Ung, Guillaume Habault, Roberto Legaspi, Hao Niu, Lian Cao, Masato Taya
Abstract
Rapid and accurate post-disaster building damage assessment is essential, yet remains a challenging task. Unmanned Aerial Vehicle (UAV) imagery offers a timely and high-resolution view of affected areas, but existing Computer Vision (CV) models often demand large annotated datasets, generalize poorly across geographic regions and their assessment policies, and are confined to the specific tasks they were trained for. Large Vision-Language Models (LVLMs) offer a promising alternative through their strong reasoning and generalization capabilities, but fall short on precise, low-level perception tasks such as object detection and accurate bounding box generation. Furthermore, they often require a substantial amount of data for effective fine-tuning on domain-specific tasks. In this paper, we propose a hybrid framework that decouples detection from damage assessment, combining the precision of CV models with the reasoning power of LVLMs. A CV model first detects buildings and generates bounding boxes on the image that are then passed to an LVLM for damage classification and contextual interpretation. We evaluated our framework on two real-world benchmarks: RescueNet and FloodNet. In particular, the best combination under this framework accurately counts intact, partially damaged and completely destroyed buildings, surpassing isolated baselines by up to 2.1 R^2 points, while requiring only limited annotated data for the detection stage. Beyond reporting aggregate gains, we provide a detailed analysis of failure scenarios and edge cases, offering practical insights for practitioners and concrete directions for future work. Our source code and data are publicly available to the research community via the following repository: https://github.com/ungquanghuy-kddi/VLM_GDINO.git