Visible thermal face recognition tested on 900 subject gallery
VTF900: A 900-Subject Visible-Thermal Face Recognition Dataset and Benchmark
Computer Vision and Pattern Recognition
Summary
Matching thermal images of faces with regular visible-light photos is hard, especially when tests only have around 100 people, which is too small to show real-world performance. The authors created a big dataset called VTF900 with 900 people, featuring different facial expressions, poses, and things covering faces. They tested several computer vision models and found that high scores on small tests don't mean good performance on large, realistic tests, especially when faces are turned or obscured. This dataset helps evaluate and improve systems that recognize faces across different types of images.
What this means in practice
- •For security system developers: Evaluate and improve face recognition systems that must identify people using thermal cameras with large subject galleries and realistic conditions.
- •For automotive safety engineers: Test in-vehicle face recognition that needs to identify drivers or passengers under varied lighting and pose using combined thermal and visible sensors.
Authors
Jacob Abaare, Bianca Ianetta, Cedric Nimpa Fondje, Shuowen Hu, Benjamin S. Riggan
Abstract
Despite advancements in visible-thermal face recognition, where thermal probe (query) face images are matched against visible enrollment templates, existing benchmarks are currently limited by small gallery sizes (N $\approx$ 100 subjects). This limitation inhibits the ability to accurately evaluate visible-thermal facial recognition at scale and to report significant recognition performance metrics, which have better operational relevance. Therefore, we introduce VTF900, a large-scale visible-thermal face recognition benchmark with 900 subjects and paired imagery that includes expression, pose, and occlusion variations. VTF900 is designed to support large-scale (i.e., large gallery) evaluations across verification (1:1 matching) and closed-set identification (1:N matching) for visible-thermal face recognition. Furthermore, VTF900 specifically addresses the challenging combination of cross-spectral and cross-dataset analysis. In this paper, after training models on the ARL-VTF dataset, we provide benchmark analyses across convolutional (baseline), vision transformers, and domain-adaptive neural network architectures to assess how these methods scale. At full gallery scale (N = 900), transformer models achieve strong discriminative performance under frontal conditions (e.g., ViT-B: 97.68% AUC, 51.10% TAR@1%FAR), while domain-adaptive methods such as DPIT-ResNet50 demonstrate greater robustness under off-pose conditions (94.68%AUC, 36.08% TAR@1%FAR). Our results show that high AUC at small scale does not translate to reliable performance at deployment scale, and that pose variation significantly amplifies performance degradation. VTF900 exposes these limitations, enabling more rigorous evaluation of cross-spectral face recognition systems.