DecoyFace: Beyond Obfuscation via Controllable and Imperceptible Identity Misdirection for Privacy-Preserving Face Recognition
2026-07-20 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors address the problem of protecting face data when part of the recognition process happens on a server that might try to misuse the information. They introduce DecoyFace, a method that mixes real face features with fake ones in a way that makes it hard to reconstruct the true face from stolen data while still allowing accurate recognition. The server uses a special step to filter out fake clues and correctly identify the face. Their experiments show this approach keeps recognition accurate and makes reconstructed fake faces look real but incorrect, greatly reducing the chance of revealing the true identity.
split face recognitionfeature inversion attackshonest-but-curious serversprivacy-preserving methodsintermediate facial featuresreconstruction-sensitive subspaceidentity leakagecanonicalizationU-Net attacksFlow-Matching attacks
Authors
Zhihan Ren, Lijun He, Xinyao Wang, Xinzhu Fu, Fan Li
Abstract
Split face recognition reduces client-side computation but exposes intermediate features to feature inversion attacks and unauthorized analysis by honest-but-curious (HBC) servers. Existing privacy-preserving face recognition methods mainly aim to resist unauthorized reconstruction, typically producing features whose inversion yields visibly degraded results, which may reveal the existence of protection and motivate adaptive attacks. To address this issue, we propose DecoyFace, an imperceptible decoy-oriented framework that steers unauthorized reconstruction toward a plausible but incorrect identity while preserving recognition utility. The key idea is to decompose the intermediate representation into a reconstruction-sensitive subspace and its complementary subspace. The client injects decoy identity cues into the reconstruction-sensitive subspace, while limited recognition-relevant evidence from the true sample is retained in the complementary subspace. On the server side, an authorized canonicalization module suppresses decoy-dominant components and recovers a recognition-friendly representation. This design addresses both attacker-side inversion from intercepted features and HBC server-side reconstruction from canonicalized representations. Experiments show that DecoyFace preserves competitive recognition accuracy while substantially reducing identity leakage to 2.93% under U-Net attacks and 0.74% under Flow-Matching attacks while yielding visually plausible and imperceptible reconstructions, with over 99.78% face validity on LFW dataset.