Vision language models struggle to match vehicles across front and back views
Front-to-Back: Benchmarking Vision-Language Models for Asymmetric Cross-View Vehicle Re-Identification
Computer Vision and Pattern Recognition
Summary
Matching a vehicle seen from the front camera to the same vehicle seen later from a rear camera is hard because the views look very different. The authors created a dataset called Front2Back-ReID with real vehicle images from South Africa to test this challenge. They compared seven different vision-language models, traditional image retrieval methods, and 25 humans. The best AI model could correctly match cars about 76% of the time, while humans did much better, around 94%. This shows current AI models still struggle compared to people at matching vehicles from very different camera angles.
What this means in practice
- •For traffic monitoring teams: Improve automated tools to track the same vehicle across different camera views for traffic flow and security analysis.
- •For security operators: Assist human operators by identifying limitations of AI models in vehicle matching to better allocate manual review efforts.
Authors
Moseli Mots'oehli, Thulani Babeli
Abstract
Matching the same vehicle across front and rear cameras is difficult because the cameras do not share a view and the vehicle's appearance changes substantially. We introduce Front2Back-ReID, a benchmark of 500 manually verified vehicle handovers from 20 recording sequences in South Africa. Each example asks a model to match a vehicle highlighted in a front-camera image to the same vehicle among at least three candidates in a later rear-camera image. We evaluate seven zero-shot vision-language models, four image-retrieval baselines, and 25 human participants. Models are tested using full front RGB images, cropped target vehicles, and binary silhouettes. The strongest VLM achieved 76.6 percent Rank-1 accuracy on target crops, compared with 74.0 percent for the frozen SigLIP2 baseline; this difference was not statistically clear. Human participants achieved 94.0 percent accuracy with full images and 92.2 percent with target crops. Under our evaluation setup, enabling reasoning improved accuracy across all three input conditions for every model evaluated in both modes. We also found that VLMs generally performed worse on full scenes than on target crops. These results show that general-purpose VLMs do not yet consistently outperform strong visual retrieval for front-to-rear vehicle matching, while humans remain substantially more reliable.