Model improves person search from text descriptions in real surveillance images
Synthetic-to-Real Text-Based Person Anomaly Search via Multi-Model Retrieval Fusion and Cross-Encoder Reranking
Computer Vision and Pattern Recognition
Summary
Searching for people in surveillance images by describing their appearance and behavior is hard because models are trained on fake images but tested on real ones. The authors designed a new system that combines multiple AI models to better match text descriptions with real images. They also created a way to tune the system using only synthetic data before testing it on actual surveillance footage. Their approach ranked eighth in a major challenge, showing it works well despite tricky differences between synthetic and real images.
What this means in practice
- •For security system developers: Improve person search capabilities in surveillance systems by better matching textual descriptions to real-world video images despite synthetic-to-real gaps.
- •For video analytics engineers: Develop enhanced retrieval systems for identifying individuals in crowded or complex scenes using multi-model fusion and reranking techniques.
Authors
Eugene Denteh, Blessing Agyei Kyem, Andrews Danyo, Joshua Kofi Asamoah, Richard Dzinyela, Solomon Owusu-Ansah, Dickson G. Asiedu, Armstrong Aboah
Abstract
Text-based person anomaly search requires a model to retrieve individuals from a large image gallery using natural-language descriptions of both appearance and behavior. This task is challenging because model training relies on synthetic image-text pairs, whereas evaluation is performed on real surveillance images containing domain differences, near-duplicate frames, and many visually similar distractors. We present a multi-model retrieval and reranking framework designed to improve synthetic-to-real transfer. The framework first adapts Qwen3-VL-Embedding-8B to synthetic person descriptions using bidirectional contrastive learning, explicit negative sampling, false-negative masking, and domain randomization. It then combines the normalized scores of six complementary Qwen3-VL checkpoints with those of a frozen SigLIP2 model using four-view gallery encoding. The fused retrieval scores identify the strongest candidates, which are jointly evaluated with the query using Qwen3-VL-Reranker-8B. The cross-encoder and retrieval scores are subsequently blended to produce the final ranking. We also construct a distractor-matched validation protocol entirely from synthetic data to select checkpoint weights, fusion coefficients, candidate depths, and reranking parameters without accessing the official test images. The final system achieves 94.7815% mAP and 90.8493% R@1 under the official AI City Challenge 2026 Track 4 test setting, with a reported eighth-place finish on the public leaderboard.