Zero knowledge proves training results of large language models quickly
zkLLMPoT: Efficient Zero Knowledge Proof of Training for Large Language Models
Artificial Intelligence
Summary
Checking if a large language model was trained as claimed is hard when weights and data are private, and fully proving the entire training is too costly. The authors created zkLLMPoT, which verifies training results by testing the model's output on selected examples instead of retracing all training steps. This method keeps model details secret and proves important performance measures efficiently, regardless of training length. It works across several model sizes and can confirm specific tasks without revealing sensitive information.
What this means in practice
- •For machine learning engineers: Verify that confidential large language models achieve claimed performance without revealing internal details or training data.
- •For security auditors: Certify properties of proprietary language models through challenge-response proofs that avoid exposing model weights or training sets.
Authors
Junkai Liang, Zhanpeng Guo, Pengfei Wu, Qingni Shen, Jiaheng Zhang, Zhonghai Wu, Haiyang Xue, Shengfang Zhai
Abstract
Auditing the claimed outcomes of large language model (LLM) training is challenging when model weights and training data are private, while cryptographically proving the full training process is prohibitively expensive at Transformer scale. We present zkLLMPoT, a zero-knowledge framework that certifies auditor-defined properties of a trained checkpoint through forward evaluation rather than verification of its optimization trajectory. zkLLMPoT includes 2 phases: 1) The trainer fixes the architecture and the model weights are committed. Then the auditor selects challenge sequences, preventing the trainer from modifying the checkpoint in response to the audit data. 2) Then the trainer proves the objective value attained by the committed model on those sequences. This formulation makes the certification cost independent of the number of training iterations, without revealing model weights or requiring access to private training data. We build on sumcheck- and lookup-based arguments to certify Transformer computations, while supporting next-token loss and task-specific audit objectives. Across four model families, operator-level benchmarks yield proving times of 41-59 seconds for 1.1-1.5B-parameter models and 131 seconds at 13B for the covered operators, with verification below half a second at a sequence length of 512.