Private evolution improves synthetic data with learning models
Understanding Private Evolution as Learning-Augmented Clustering
Machine LearningCryptography and Security
Summary
Generating synthetic data privately can be tricky, especially when trying to keep data secure and accurate. The authors look at a method called Private Evolution (PE), which creates private synthetic data but sometimes struggles on simple clustered groups. They explain why PE works better when combined with a generator that understands the data's true shape, reducing the number of samples needed. They also propose a new version of PE that reliably converges on well-structured data and shows improved performance in tests.
What this means in practice
- •For data privacy engineers: Improve synthetic data generation tools by integrating geometry-aware private evolution to enhance accuracy and convergence on clustered data.
- •For machine learning platform developers: Incorporate generative model-augmented Wasserstein learning to reduce sample requirements for private synthetic data creation in low-dimensional settings.
Authors
Audra McMillan, Kunal Talwar, Felix Zhou
Abstract
Private Evolution (PE) is a differentially private algorithm for synthetic data generation. While it can be viewed as a Wasserstein learning algorithm, it performs much better in practice than worst-case Wasserstein analyses would predict. We recast PE as generative model-augmented Wasserstein learning. We show theoretically that when we take into account the use of a generative model that is able to capture something about the true distribution, then we can obtain much better performance bounds. For example, if the generator gives samples in the same low-dimensional space as the distribution, then sample complexity depends on intrinsic, not ambient, dimension. We also show that standard variants of PE can fail to converge on simple well-clustered instances, and propose a new geometry-aware version of PE with provable convergence on such instances. Experimentally, we show that our new algorithm is competitive with standard baselines and can improve recall.