Global average precision improves ranking and retrieval consistency
Global Average Precision for Representation Learning
Computer Vision and Pattern RecognitionMachine Learning
Summary
When computers learn to find things like images or text, they usually decide how good results are one search at a time, ignoring how those scores compare between searches. This paper introduces a way to measure and improve the quality of results by looking at all searches together, making it easier to pick a single scoring cutoff that works well everywhere. The authors also provide a new method to train computers with this improved measurement, which helps them retrieve more relevant items and improve related tasks like recognizing new things without extra training. This approach works with many different kinds of data and learning settings.
What this means in practice
- •For machine learning engineers: Improve metric learning models by replacing per-query losses with a global ranking loss that enhances retrieval consistency and accuracy under a single threshold.
- •For computer vision teams: Train self-supervised pretraining and cross-modal alignment models more effectively using a loss that jointly considers all query-candidate pairs, boosting transfer and zero-shot recognition.
Authors
Bill Psomas, Mohammad Mahdi, Michalis Thomas, Danda Pani Paudel, Giorgos Tolias, Giorgos Kordopatis-Zilos
Abstract
Standard information retrieval metrics, such as mean Average Precision (mAP), assess performance one query at a time, based on how the similarities between a query and its positives compare against those with its negatives. The same holds for common representation learning losses, such as InfoNCE and per-query AP surrogates. None of them considers whether similarities are comparable across queries, which any system with a single decision threshold relies on. Global Average Precision (gAP) does, by ranking all query-candidate pairs in one list and computing a single AP. We introduce gSAP, a differentiable surrogate of gAP. It needs only a similarity matrix and a binary matrix marking the positive pairs, the same input as existing losses, so it is a drop-in replacement for them and agnostic to the encoder, the modality, and the source of supervision. Since it considers all possible pairwise comparisons in the batch jointly, it also remains trainable at low temperatures, a regime where per-query surrogates run out of gradient. Swapping it into established recipes improves supervised metric learning, cross-modal alignment, and self-supervised pretraining, where, to our knowledge, it is the first ranking loss to replace the community standard InfoNCE in the latter two. Its similarities are more consistent across queries, which drives the gains under a universal threshold. gSAP retrieves up to four times as many positive pairs as the strongest AP surrogate at the same precision, and it degrades the least when queries with no positives in the database are added. Beyond thresholding, models trained with gSAP also learn better representations, with higher transfer, $k$NN and zero-shot classification accuracy.