Ranking vs. Assignment: The Metric Mismatch in Multi-View Object Association
2026-06-01 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial IntelligenceMachine Learning
AI summaryⓘ
The authors study how to connect objects seen from multiple cameras, which is usually done by matching them one-to-one. They point out that commonly used scores (AP and FPR-95) to judge model performance don’t always reflect the true quality of these matches. They prove that even perfect matches can have imperfect scores unless a specific normalization method (Sinkhorn-based) is applied. Through experiments, the authors show that improving these scores with post-processing does not always mean better actual matching results.
multi-view object associationone-to-one matchingaverage precision (AP)false positive rate (FPR-95)Sinkhorn normalizationpairwise rankingassignment problemcomputer vision evaluation metrics
Authors
Matvei Shelukhan, Timur Mamedov, Aleksandr Chukhrov, Karina Kvanchiani
Abstract
Multi-view object association is an important computer vision problem that underlies many multi-camera perception tasks. While this task is naturally formulated as a constrained one-to-one matching problem, recent works heavily rely on pairwise ranking metrics like AP and FPR-95 for model evaluation. We highlight a fundamental mismatch between these metrics and the actual assignment objective. Theoretically, we show that AP and FPR-95 can be imperfect even when the assignment is already correct, and that Sinkhorn-based normalization can make them perfect. Conversely, optimal pairwise ranking can still lead to incorrect assignments. We validate this mismatch in practice by using our Sinkhorn-based normalization as a controlled post-processing stress test. We show that optimizing just a few post-processing parameters significantly boosts AP and FPR-95 without corresponding improvements in assignment-level metrics such as ACC and IPAA.