Improved method detects mixed watermarks in AI-generated text accurately
Token-Level Attribution of Multi-Mode LLM Watermarks: Optimal Rule and Belief Weighting
Cryptography and Security
Summary
Sometimes, AI-generated text can include secret watermarks from different sources mixed together, making it tricky to detect which watermark is present. The authors studied how to best identify these mixed watermarks at the level of individual words or tokens. They developed a new method called BLADE that estimates which watermark mode is most likely for each token, improving accuracy without increasing false alarms. Their approach works well on popular language models, detecting watermarks more reliably across different scenarios.
What this means in practice
- •For content moderation teams: Identify and attribute mixed watermarks in AI-generated texts to determine their source accurately.
- •For digital forensics analysts: Use token-level watermark detection to verify authenticity and origin of suspicious documents with mixed watermarks.
Authors
Rui Zhou, Licheng Zhao, Chao Shen, Wenqiang Pu
Abstract
A watermark verifier may hold many modes, i.e., secret keys, and a document may mix spans of different modes with unwatermarked text. Directly reporting the mode with the largest pivotal statistic (pivot) inflates the Type I error, and restoring the nominal level reduces power as candidate modes are added. We formulate token-level multi-mode attribution for the Gumbel-max watermark as maximizing correct-label power under a Type I error constraint given a mode prior. We show that the optimal rule reports the mode with the largest prior-weighted alternative density and rejects when this density exceeds one common threshold. Based on this rule, we propose the Belief-weighted Likelihood Attribution with Density Estimation (BLADE) detector. It estimates the mode prior from neighboring pivots and the alternative density from reference token probabilities. Under a conditional product model of the pivots, it keeps the exact Type I error. We further show that BLADE approaches the optimal rule as its prior and density estimates improve, with a sharp coefficient on the density error. On OPT-1.3B and other open-weight large language models, BLADE attains higher correct-label power than the compared detectors across candidate counts, token-substitution rates, generation temperatures and generator models.