Task-driven image zooming improves object detection accuracy and speed
Task-driven Processing with Coarse-to-Fine Glimpse-based Active Perception
Computer Vision and Pattern Recognition
Summary
Many modern computer vision systems look at entire images at once, which can miss important details, especially when trying to spot specific objects in large, cluttered pictures. The authors propose a method called CF-GAP that helps the system look at the image step-by-step, zooming in on important parts using clues from the task it is doing. This approach allows the system to process only the relevant high-resolution spots instead of the whole image, saving computing power and improving accuracy. Tests show that CF-GAP can make object detection better by up to 20% and let smaller, faster detectors work better than bigger ones.
instance detectionhigh-resolution imagesactive perceptioncoarse-to-fine processingglimpse-based attentionaverage precisioncomputer visiontask-driven processing
Authors
Oleh Kolner, Thomas Ortner, Stanisław Woźniak, Angeliki Pantazi
Abstract
State-of-the-art vision models process images in their entirety, lacking the ability to selectively zoom in on relevant regions. This limitation is particularly acute in scenarios where processing must be conditioned on a specific task - such as instance detection, which requires localizing a specific object in a high-resolution, cluttered scene. In such settings, critical details are easily lost as images are often resized to match the model dimensions and computational constraints. We introduce Coarse-to-Fine Glimpse-based Active Perception (CF-GAP), a task-driven front-end that enhances high-resolution processing of existing instance detectors. CF-GAP selectively directs a sequence of limited view glimpses across the scene, utilizing task information to iteratively refine focus on the most relevant regions. These localized regions are then processed at high resolution by a downstream instance detector. By avoiding full-image processing and eliminating irrelevant confounding information, CF-GAP improves Average Precision (AP) by up to 20% across various state-of-the-art instance detectors on the HR-InsDet and Robotools benchmarks, while further enabling lightweight detectors to outperform their larger counterparts.