Watermarking improves language model text detection with less quality loss
Watermarkability: Measuring and Improving Language-Model Compatibility with Logit-Based Watermarks
Computation and LanguageCryptography and Security
Summary
It can be hard to add invisible markers to AI-generated text without messing up how natural the text sounds. The authors introduce a way to measure how easily a language model can accept these markers, called watermarkability. They train small add-ons to the model that help shift probabilities toward certain words, so it’s easier to insert these markers with less impact on text quality. Their approach leads to more reliable watermarking and less decrease in language fluency.
What this means in practice
- •For content moderation teams: Identify AI-generated text more reliably in diverse contexts by embedding detectable watermarks with minimal quality loss.
- •For ai platform operators: Improve provenance tracking and detect unauthorized edits by integrating watermark adapters that maintain text fluency.
Authors
Zewei Deng, Mohamed Seif, Liyan Xie
Abstract
Language-model watermarks can help identify model-generated text, encode provenance information, and detect post-generation edits. A broad class of logit-based watermarks increases the logits of a designated token set at each decoding step. Because a base model's native probability mass on this set varies across contexts, models, and watermark rules, the same logit bias can yield different adherence rates and increases in perplexity (PPL). We introduce watermarkability to characterize the compatibility between a model and the token sets requested by a watermark. At a target mass $τ$, we quantify this property through a watermarkability cost: the expected minimum KL divergence from the base-model distribution required to assign probability at least $τ$ to the requested set. A lower cost means that the model is more watermarkable for that request distribution. To reduce this cost, we propose a two-stage method. In the first stage, we train a target-set-conditioned low-rank adapter on unions drawn from a 16-way vocabulary partition to increase the requested set's probability mass. In the second stage, a closed-form controller applies the smallest remaining uniform logit bias needed to reach the target mass. We evaluate the method on six watermark schedules using OPT-125M and Qwen3-8B. Across the 12 model-schedule pairs, the pre-adaptation watermarkability cost correlates with the log-PPL increase under a near-hard fixed-bias baseline (pooled Pearson $r=0.929$). For every schedule, at least 99.80% of sampled tokens fall in the target set requested at their decoding step. At this reliability, the adapter and controller reduce average per-schedule prompt-conditional PPL by 46.3% on OPT-125M and 5.2% on Qwen3-8B relative to the fixed-bias baseline.