Statistical methods assess probabilistic ai model identity remotely
Characterizing Statistical Separability in TP-CRIV for Probabilistic AI Models
Cryptography and SecurityArtificial Intelligence
Summary
It can be hard to prove that a remote AI model is exactly the same as a local one, especially when the model gives different answers to the same question each time. The authors study how much and what kind of information is needed to reliably check if two models match despite this randomness. They develop a statistical framework that connects the patterns of responses to how well the verification works overall. They test this method using large language models and show it can estimate how many checks are needed for a good decision.
What this means in practice
- •For security engineers: Verify that deployed AI models remotely accessed are authentic despite output randomness, ensuring trustworthy model identity checks.
- •For ai system integrators: Determine the amount of interaction needed to reliably confirm remote probabilistic AI models match expected versions before integration.
Authors
Teruki Sano, Minoru Kuribayashi, Masao Sakai, Shuji Isobe, Eisuke Koizumi, Zhang Zhang, Satoru Matsumoto
Abstract
Third-party challenge-response identity verification (TP-CRIV) enables an independent verifier to assess whether a claimant possesses a model identical to a remotely deployed model without directly accessing the reference model. However, for probabilistic AI models, repeated executions of the same query may produce different outputs and therefore different verification observations. This raises the question of how such stochastic evidence should be accumulated and how much evidence is required for reliable verification. In this work, we characterize statistical separability in TP-CRIV of probabilistic AI models. Specifically, we relate challenge-wise behavior of matching and non-matching provers to verification-level separability. The characterization explicitly describes how the numbers of independent challenges and repeated responses affect detection performance and enables the verification budget required for a target AUC to be estimated. We instantiate the proposed characterization for LLMs using open-ended challenges. The experiments demonstrate matching-non-matching separation, close agreement between theoretical and empirical AUCs, and consistent estimates of the minimum verification budgets. These results provide a statistical basis for relating probabilistic model behavior to verification-level separability and the evidence required for third-party verification.