Scalable deep feature encoding boosts scene image retrieval accuracy
Rethinking Fisher Vectors for Second-order Multi-scale Deep Feature Aggregation in Scalable Scene Retrieval
Artificial Intelligence
Summary
Finding similar photos in large collections is hard because computers struggle with changes in viewpoint and scale. The authors introduce a new way to combine information from multiple scales of deep neural network features, capturing richer details without relying on predefined codebooks. They encode this information using a method related to Fisher Vectors, improving how images are compared and retrieved. Tests on challenging image datasets show their approach outperforms previous methods with only a small increase in data size.
What this means in practice
- •For image retrieval engineers: Improve scene image search engines by encoding multi-scale deep features without needing codebook construction, leading to higher accuracy and scalable retrieval.
- •For digital asset managers: Enhance organization and finding of photos by using richer image feature representations that better capture scene details across different scales and viewpoints.
Authors
Alexy Bhowmick, Sarat Saharia, Shyamanta M. Hazarika
Abstract
Aggregating CNN responses into a compact discriminative vector, given their lack of geometric invariance, is a challenge. To address this challenge, we propose a codebook-free method to encode second-order information of multi-scale CNN features within the Fisher Kernel framework. We use a scalable retrieval pipeline to show the efficacy of the proposed vector. The increased discriminative power is confirmed by experimental evaluation of our method on two difficult benchmarks (INRIA Holidays, UKBench). The significant improvement in retrieval accuracy over state-of-the-art approaches comes with only a slight dimension increase.