BiHashFormer improves object detection in high-resolution wide images
BiHashFormer: Hash-Driven Dual-Branch Transformer for Efficient Object Detection in HRW Shots
Computer Vision and Pattern Recognition
Summary
Detecting objects in extremely large, detailed images is hard because objects are sparse and surrounded by lots of background. The authors introduce BiHashFormer, a method that uses two parallel parts working together to focus on important areas and reduce unnecessary processing. One part zooms in on likely areas with objects, while the other checks the rest efficiently to recover any missed spots. This approach makes detection faster and more accurate on huge images, like gigapixel photos.
What this means in practice
- •For remote sensing analysts: Detect small objects in ultra-high-resolution satellite or aerial imagery faster and more accurately by processing likely regions in parallel with efficient background checks.
- •For surveillance operators: Improve real-time detection of sparse targets in wide-area security camera feeds by reducing processing delays and focusing attention efficiently.
Tested on one dataset.
Authors
Jingchen Huang, Wenxi Li, Chenyang Lyu, Moran Liu, Haozhe Lin, Yuchen Guo
Abstract
Recent advances in gigapixel-level imaging have brought High-Resolution Wide shots to the forefront of research. However, these images present significant challenges: extreme sparsity of foreground, gigapixel-level resolutions and interleaving of foreground and background. This causes traditional detectors and attention mechanisms to be hindered by background, resulting in inefficiency and inaccuracy. To tackle this problem, we propose BiHashFormer, a dual sparsified branch transformer built on hashing. RoI Selector will predict the foreground proportion for top-k selecting target-containing windows. The compression branch processes windows selected and refines attention into HashAttention by discarding half of the key-value pairs, reducing cost and improving efficiency. The compensation branch uses HashMiner to perform low-cost hash searches in the remaining regions, recovering features of missed objects. Experiments confirm that the selection relationship in HashAttention is one-way and reveal the preference of different queries for keys. In experiments on the gigapixel benchmark PANDA, BiHashFormer reduces 37.5% backbone FLOPs while improving AP50 to 81.0%. Moreover, this parallel dual-branch structure allows the model to avoid inference delays caused by inter-branch dependencies, while achieving much greater stability than single-branch models, specially with low rates of RoI selection.