Discard the Dross and Select the Essential: Pre-query Sample Selection for Black-box Membership Inference Attacks
2026-06-29 • Cryptography and Security
Cryptography and Security
AI summaryⓘ
The authors study how to improve black-box membership inference attacks, which try to tell if certain data were used to train a model by asking the model questions. They note that not all data give clear clues, so asking indiscriminately wastes effort and risks revealing the attack. To fix this, they propose PSS-MIA, a method that first picks the most informative samples based on a new ranking technique called Loss-Gap Ranking before querying the model. Their tests show that this approach reduces the number of needed queries by a lot while maintaining good attack performance across different datasets.
membership inference attackblack-box modelquery costmembership signalsample selectionLoss-Gap Rankingreference modelsfalse positive rateCIFAR-10CIFAR-100
Authors
Dongdong Zhao, Jinrong Hu, Changtian Song, Jian Chen, Hongmin Wang, Baogang Song
Abstract
Black-box membership inference attacks (MIAs) rely on target-model queries to infer whether candidate samples were used for training. However, membership signals are highly non-uniform across samples: some candidate samples support strong member/non-member separability, whereas many others provide little useful signal. Consequently, indiscriminate querying can incur substantial query cost and increase query-induced exposure, with limited marginal benefit for inference. This raises a key question: which candidate samples are worth querying for black-box MIAs? To address this question, we propose PSS-MIA, a pre-query sample selection framework which can be embedded with any existing MIA methods. PSS-MIA proceeds in two stages: it first ranks candidate samples and selects a subset expected to support stronger membership inference, then queries the selected samples and uses the returned outputs for an existing black-box MIA, thereby reducing query cost and query-induced exposure. In the first stage, we propose Loss-Gap Ranking (LGR), which ranks candidate samples by estimating the strength of their membership signal using loss gaps computed from reference models. Experiments on CIFAR-10, CIFAR-100, and CINIC-10 with five representative black-box MIA methods demonstrate that PSS-MIA with LGR consistently outperforms all other compared methods. Moreover, under a 0.1% FPR constraint, PSS-MIA can save at least 83.1%, 60.6%, and 80.4% of the query budget for the three datasets, respectively.