Zero knowledge friendly quantization improves private large language model checks
Bits Under ZK-LLM: Evaluating Zero-Knowledge-Friendly Quantization for Verifiable Private LLM Inference
Artificial Intelligence
Summary
Checking that large language models (LLMs) behave properly without revealing their secret parts is hard. The authors study how changing the way these models do calculations affects the ability to prove their correctness privately using zero-knowledge proofs. They find that some aspects of the models, like activation precision and lookup operations, are much more sensitive and affect accuracy more than others. They also show that simply reducing the calculation precision does not always save verification time, suggesting smarter methods are needed.
What this means in practice
- •For model deployment teams: Optimize model quantization settings to balance verification cost and accuracy during private zero-knowledge proof-based inference.
- •For compliance auditors: Use zero-knowledge-friendly quantization methods to verify LLM inference behavior without exposing proprietary model details.
Authors
Taeung Yoon, Yupeng Zhang, Xiaojing Liao
Abstract
Zero-knowledge proofs are emerging as a promising approach for enabling private, verifiable LLM governance and auditing, where regulators, users, and auditors need to verify claims about training-data usage or LLM inference-time behavior, while model providers must protect proprietary model parameters. However, despite the growing interest in ZK-LLMs, the understanding of ZK-friendly quantization remains limited. This gap matters because in the ZK setting, quantization directly shapes the arithmetic structure, constraint complexity, and proving cost of ZK inference. ZK protocols operate over finite fields and incur costs that depend heavily on the number and type of arithmetic operations, nonlinearities, and lookup constraints. Understanding ZK-friendly quantization is therefore essential for making ZK-LLMs practical. In this work, we present the first systematic study of ZK-friendly quantization for LLMs. We first formalize the definition of ZK-friendly quantization, capturing the properties required for ZK proof generation. We then evaluate nine language models, including Qwen2.5-14B and the mixture-of-experts model Qwen3-30B-A3B, across a broad design space of weight, activation, and nonlinear lookup table precision. Our results show that activation precision is substantially more sensitive than weight precision, while nonlinear lookup approximations can become the dominant source of utility degradation. Also, we identify RMSNorm inverse-square-root lookups as a recurring bottleneck in several large models and recover near-baseline utility by selectively increasing precision only at the bottleneck. Finally, we show that reducing bit-width or lookup-table size does not necessarily yield proportional end-to-end proving savings, showing that conventional low-bit quantization heuristics do not directly translate to ZK proving efficiency and motivating operator-aware precision selection.