Adaptive synthetic data improves face recognition under hard conditions
Adaptive Adversarial Augmentation for Controllable Face Synthesis
Computer Vision and Pattern Recognition
Summary
Face recognition systems need lots of different pictures to work well, especially when faces are blurry or masked. The authors created a way to make realistic but diverse fake face images that are challenging for the computer to recognize. Training with these images helps face recognition work better in real life. They also explain how to balance making images hard enough to learn from without confusing the system too much.
What this means in practice
- •For security system developers: Enhance face recognition models to handle low-quality and occluded faces by training with challenging synthetic images.$Commercial implications: Enables selling more robust security and access control systems that work well despite masks or poor lighting.
- •For mobile app developers: Improve face unlocking features by using diverse synthetic training data to boost recognition under varied real-world conditions.
Authors
Saransh Suri, Shivang Agarwal, Mayank Vatsa, Richa Singh
Abstract
Synthetic data provides a scalable alternative to real-world datasets for training face recognition models, particularly under challenging conditions such as low resolution, occlusion, and masks. Yet, most approaches lack diversity and fail to generalize effectively. We propose Ensemble Feedback Controllable Synthesis (EFCS), a guided framework that generates diverse and challenging samples while preserving visual realism. EFCS expands distributional variability, often reflected in higher FID and KID scores compared to single-feedback and random synthesis, while maintaining high precision. Recognition models trained on EFCS data consistently outperform baselines across multiple benchmarks, showing improved generalization to real-world scenarios. Furthermore, we introduce an analytically motivated formulation linking perturbation-induced difficulty, sample utility, and performance degradation, offering principled insights into balancing synthetic data complexity for optimal training. Together, these contributions establish EFCS as an effective and analytically grounded approach for bridging the gap between synthetic and real datasets.