Dynamic budgeting improves tiny object detection speed and accuracy

Dynamic Query as Budgeting for Tiny Object Detection

Computer Vision and Pattern Recognition

Summary

Detecting tiny objects in images is hard because there are often many more objects than a detector's queries can handle. The authors found that what really matters is deciding how many queries to use based on object density, not which specific queries to select. They designed a simple method called BUTR that adjusts the number of queries automatically using a lightweight budgeter, leading to faster and more accurate detection. This approach works well on tiny object datasets and general detection tasks without adding complexity.

What this means in practice

  • For autonomous vehicle engineers: Improve detection of tiny objects like distant pedestrians or obstacles by dynamically allocating detector queries based on scene density to balance accuracy and speed.
  • For drone surveillance teams: Enhance real-time identification of small objects in aerial footage by deploying the lightweight budgeting approach to adjust query numbers adaptive to scene complexity.
  • For security camera developers: Integrate adaptive query budgeting in object detectors to better detect small and densely packed objects in crowded scenes without extra computational overhead.$Commercial implications: Enables sale of more efficient and accurate surveillance analytics products by reducing computational costs while improving detection quality.

Authors

Yi Zhang, Di Xiong, Ge Gao, Xiangyue Zhang, Yihang Qiu, Yuxuan Zhou, Shuo Chen

Abstract

Recent DETR-based tiny object detectors adopt dynamic query mechanisms to handle severe density imbalance. However, existing designs entangle two effects: how many and which queries are decoded. This entanglement obscures what drives the gains and complicates dynamic-query design; additional learned predictors and heuristic filtering also introduce overhead and potential selection errors. We disentangle these effects and show that dynamic query primarily acts as \emph{budgeting}---allocating decoder query capacity according to object density---rather than intricate query selection.Motivated by this view, we formulate a monotonic and conservative budgeting principle and propose a budgeting-based DETR (BUTR), a streamlined framework that uses a lightweight, non-learned budgeter to estimate an input-dependent query budget $K(x)$ from encoder confidences. BUTR retains standard Top-$K(x)$ query initialization without additional learned modules or heuristic filtering in the budgeter.At $800\times800$ resolution, BUTR-L achieves 35.7 AP at 47.8 FPS on AI-TOD-v2 and 29.8 AP at 42.1 FPS on VisDrone on an NVIDIA T4 GPU. Inference-time integration on COCO further demonstrates its applicability to general scenario detection. Code is available at https://github.com/zhangyi-magic/BUTR.