Privacy improvement for machine learning with fixed batch sizes
Counting on Dummies: Privacy Amplification with Observable Counts and Truncated Batches
Machine LearningCryptography and Security
Summary
When training machine learning models privately, data is processed in groups called batches. Usually, these batches are randomly chosen for better privacy, but revealing their sizes can leak information. The authors propose adding fake data points to hide the true batch size, even when batches have a maximum size limit. This method keeps privacy strong while being practical for real-world use, especially in systems where batch sizes might get cut off or seen by others.
What this means in practice
- •For machine learning engineers: Maintain strong privacy guarantees when training models with fixed-size data batches in distributed or centralized environments.
- •For data privacy officers: Better measure and control privacy risks in systems that reveal batch sizes during private data aggregation.
Authors
Vitaly Feldman, Audra McMillan, Kunal Talwar
Abstract
In many practical deployments of Differentially Private Stochastic Gradient Descent (DP-SGD), the server processes data in batches. Randomly sampling these batches allows for better privacy bounds via privacy amplification results. Most sampling schemes with provable privacy amplification, including Poisson sampling and random allocation, result in variable batch sizes. The privacy guarantee of these schemes requires that the batch sizes are not revealed. In distributed settings, the server performing the aggregation or any party observing the communication traffic may learn these batch sizes. Furthermore, in both centralized and distributed settings, memory and computation constraints may require each batch to be of some maximum size $s$, necessitating truncation of the batch when it exceeds this limit. It is known that this truncation can result in a significant increase in privacy loss. In this work, we address both of these challenges via a simple mechanism that uses a random number of \emph{dummy points} to privatize the batch size and maintain the privacy guarantee even under truncation. We demonstrate that our mechanism is easy to implement in a distributed setting, and provide a general analysis of this mechanism for both Poisson sampling and random allocation. Our analysis is designed to support efficient and relatively tight privacy accounting. It also effectively amplifies the privacy of the count releases by subsampling, making the expected number of dummy points asymptotically smaller than what is needed to directly privatize the (sensitivity-one) batch sizes with non-negative noise. For Poisson sampling with truncated batches, we demonstrate empirically that our mechanism achieves better privacy--efficiency tradeoffs than the direct accounting for truncation recently given by (Ganesh, 2025).