Membership inference attack reveals privacy risks in synthetic data generation
Membership Inference Attack on Private Evolution
Cryptography and Security
Summary
Sharing private data safely is tricky, so people use synthetic data that looks real but doesn't expose secrets. Private Evolution is a method that tries to create such synthetic data with privacy protections. This paper shows that even with these protections, it's possible to guess if someone's data was used by looking carefully at how the synthetic data was made. The authors created an attack that can see signs of private data being leaked in images, text, and tables generated this way.
What this means in practice
- •For privacy engineers: Evaluate synthetic data generation tools for vulnerabilities to membership inference attacks specific to voting-based algorithms like Private Evolution.
- •For data protection officers: Assess the risks of releasing synthetic datasets created with differential privacy methods under practical membership inference threats.
Authors
Daniil Filienko, Sikha Pentyala, Martine De Cock
Abstract
Synthetic Data Generation (SDG) is an actively studied way to enable safe sharing of private data more broadly. Inference-only SDG methods, such as Private Evolution (PE), are particularly appealing, as they can leverage black-box access to pre-trained foundation models to generate high quality data without the need for model fine-tuning on the private data. PE is explicitly designed to produce synthetic data under Differential Privacy (DP). Yet formal DP guarantees do not eliminate the need for empirical privacy auditing of the practical susceptibility of released synthetic data to adversarial attacks. In this work, we introduce the first white-box Membership Inference Attack (MIA) tailored to the voting-based algorithm underlying the PE framework. By exploiting how the iterative filtering process biases the survival and proximity of synthetic candidates toward private instances, our attack achieves empirical performance indicating private data leakage in the PE applications across image, text, and tabular modalities.