Zero knowledge AI output verification requires cryptographic oracles
Can AI Oversight Be Zero Knowledge?
Artificial IntelligenceComputational ComplexityCryptography and Security
Summary
AI sometimes needs to check answers calculated from secret or private information without revealing that secret data. The authors found that it's generally impossible to verify these answers in a totally secret way (called zero knowledge) if the verification uses external sources, like human opinions or experiments. But if the external source signs its answers securely, then it becomes possible to verify the AI's answer without leaking anything extra. This gives a new way to make sure AI results are correct and private at the same time.
What this means in practice
- •For security engineers: Verify AI-generated outputs based on secret data without revealing any sensitive information by relying on cryptographic signatures from trusted oracles.
- •For compliance officers: Implement privacy-preserving oversight methods for AI outputs involving confidential inputs to ensure correct AI decisions without exposing secrets.
A theory result. No direct application yet.
Authors
Alessandro Chiesa, Ziyi Guan, Burcu Yildiz
Abstract
AI systems increasingly produce outputs from confidential data, such as a fitness-for-duty assessment from medical records or the predicted properties of a drug candidate from its secret structure. It is important to verify that such outputs are correct without revealing the underlying data. A recent line of work studies verification of AI outputs via interactive proofs and debate for oracle-aided computation, where correctness may depend on an oracle such as human judgment, a physical experiment, or the web. These works focus on verification by a verifier that runs much faster than the computation. However, such efficient verification is impossible for general oracle-aided computation, and these works therefore rely on additional assumptions. We focus instead on privacy: allowing the verifier to run in time polynomial in the computation, we ask whether interactive arguments for oracle-aided computation can be zero knowledge, so that the verifier learns nothing about the confidential data beyond the correctness of the output. We prove that, in general, they cannot. In the random oracle model, there are no zero-knowledge proofs for all oracle-aided computations, even if both the prover and the verifier are allowed to run much longer than the computation itself. The impossibility extends to debate, a canonical model for scalable oversight. On the positive side, we show that if the oracle attaches a cryptographic signature to each of its answers, then every oracle-aided computation can be verified in zero knowledge with an efficient prover and verifier, assuming only collision-resistant hash functions. Beyond privacy, this also gives an alternative approach to scalable oversight that relies neither on an honest opponent, as in debate, nor on the robustness of the computation, as in prior single-prover protocols.