Neighbors help find training data leaks in a single model
Calibrating One-Round Membership Inference with Neighbors
Cryptography and SecurityArtificial IntelligenceMachine Learning
Summary
Knowing if a specific data point was used to train a big AI model can reveal privacy risks. Usually, this requires extra models that skip the data point to compare behavior, but that’s too costly for large models. The authors found that instead of training extra models, looking at similar nearby points and how the single trained model treats them can give a strong signal of membership. This method works well across different image tasks and training scenarios without needing any extra training.
What this means in practice
- •For machine learning engineers: Detect if training data points are vulnerable to membership inference attacks using only the trained model and neighbor samples without extra model training.
- •For privacy auditors: Assess privacy leak risks of large AI models when only a single trained model is available by leveraging behavior on similar data points.
Authors
Francesco Rita, Jie Zhang, Florian Tramèr
Abstract
The state-of-the-art Membership Inference (MI) methods calibrate their signal separately for each example using reference models, auxiliary models trained to exclude the target. This paradigm scales poorly to modern large models, however, whose training is too expensive to replicate. This has motivated one-round settings, where only a single trained model is available; but without reference models the per-example calibration that drives the strongest attacks can no longer be estimated, leaving the membership signal weak. We ask whether neighbors of the target point can recover this calibration without training any additional model. Our key observation is that reference models serve only to reveal how an example behaves under models not trained on it, and that querying the target model on nearby samples yields the same information. We propose two complementary ways to obtain such neighbors, and show that querying them against an early training checkpoint further sharpens the signal. We evaluate across three image classification datasets and three training setups, showing that neighbors yield strong membership signals and competitive attack performance at no additional training cost.