Restoration guided by recognition task improves face identification accuracy
Restore What Matters: Lessons from Joint Restoration and Recognition
Computer Vision and Pattern Recognition
Summary
Traditional methods try to clean up images first and then identify people, but this doesn’t always help recognition. The authors suggest focusing restoration only on parts of images needed for identifying faces, guided by the recognition task itself. They combine physics-based image modeling, insights about human attention, and end-to-end learning to improve accuracy. Tests show better recognition rates and efficiency by skipping restoration when unnecessary.
What this means in practice
- •For security system engineers: Improve facial recognition accuracy in surveillance cameras affected by blur or turbulence by restoring only essential image regions informed by recognition feedback.
- •For drone operators: Reduce computational load by applying selective image enhancement for recognizing targets in aerial imagery with atmospheric distortion.
Authors
Lanqing Guo, Xijun Wang, Minchul Kim, Yu Yuan, Wes Robbins, Xingguang Zhang, Nicholas Chimitt, Stanley H. Chan, Zhangyang Wang, Xiaoming Liu
Abstract
Recognition pipelines typically adopt a restore-then-recognize workflow, yet decades of experience show that generating visually pleasing images seldom translates to improved recognition. We propose a Joint Restoration-for-Recognition (JR$^2$) paradigm: restore only what downstream tasks truly require, with task signals dictating where, how much, and whether restoration is necessary. JR$^2$ rests on three pillars: (i) Physics, employing optics-accurate turbulence simulation, extensible to blur and noise, to ground restoration in real image formation; (ii) Neuroscience, drawing on selective attention and neuroplasticity to direct model capacity toward identity-critical regions and frames while bypassing already-clean inputs; and (iii) Vision & Learning, coupling recognition loss end-to-end through restoration and alignment so that low-level edits maximize high-level identity stability. Evaluations on IARPA-BRIAR show consistent improvements (e.g., TAR@0.01% FAR +0.6; FNIR@1% FPIR -2.5), while a quality gate skips ~70% of clean frames, reducing cost. Ablations confirm physics priors enhance realism, joint training prevents catastrophic forgetting, and selective restoration suffices in many cases. We conclude that better-looking images are neither necessary nor sufficient; restoration modules must be task-driven, selective, and physically aware. Code, pretrained models, and recipes are provided for integration.