Differentially private high dimensional synthetic data with sliced moments
Differentially Private Synthetic Data from Sliced Chebyshev Moments
Machine LearningComputers and SocietyData Structures and Algorithms
Summary
Creating synthetic data that protects individual privacy is tricky, especially when the data has many features. Existing methods either focus on preserving small parts of the data or struggle with high dimensions. The authors introduce SPLICE, a new method that privately generates synthetic data matching the overall distribution without depending on the data’s complexity. They do this by measuring special mathematical summaries called sliced Chebyshev moments along random directions and use these to build synthetic data that is both private and accurate. This approach performs well in practice and comes with strong theoretical guarantees.
What this means in practice
- •For data privacy engineers: Generate high-dimensional synthetic datasets that protect individual privacy while maintaining global distribution characteristics for private data sharing.
- •For machine learning platform teams: Improve simulation and testing pipelines by creating realistic synthetic data with strong privacy guarantees independent of feature dimension count.
Authors
Lucas Rosenblatt, R. Teal Witter, Christopher Musco
Abstract
Research on practical differentially private (DP) synthetic data has largely focused on methods that approximately preserve low-dimensional marginals of a target distribution, and can thus only hope to preserve higher-dimensional, global structure as a lucky by-product. At the same time, existing methods that target global accuracy guarantees pay a price exponential in the data dimension, $d$. In this work, we address this ``curse of dimensionality'' by presenting a new DP synthetic data algorithm, SPLICE, that comes with a compelling global distributional guarantee, even in high-dimensions, and moreover, is practically state-of-the-art. Our algorithm is based on measuring noisy sliced Chebyshev polynomial moments of the target distribution and fitting a distribution to match, a twist on recent polynomial methods for 1-D synthetic data. We prove that, given a dataset of $n$ examples, SPLICE returns a private distribution with Sliced Wasserstein-1 distance $\tilde{O}(1/\sqrt{n})$ from the target, for constant privacy parameters. Notably, this bound is independent of the data dimension, $d$. Along the way, we prove a moment matching result of independent interest: for a distribution on the unit ball, matching the first $O(1/α)$ Chebyshev moments along $\tilde{O}(d/α^2)$ random directions guarantees $α$ Sliced Wasserstein error. Finally, we provide a practical implementation of the algorithm that borrows tools from the tensor decomposition literature and achieves state-of-the-art performance compared to prior DP synthetic data methods on an extensive benchmark.