ClockMark improves watermarking for large language model text generation
ClockMark: Adaptive and Host-Dependent Watermarking for Large Language Models
Machine LearningComputation and Language
Summary
Watermarking large language model outputs is tricky because each word generated changes what comes next. The authors explain that watermarking should adjust as the text unfolds and depend on the model’s current predictions. They propose ClockMark, a new way to embed hidden information in text that keeps the quality of the writing intact. Their method is better at surviving edits and changes in the text than older approaches.
What this means in practice
- •For content moderation teams: Detect AI-generated text robustly even after edits by embedding hidden markers that maintain text quality.
- •For digital content distributors: Embed persistent watermarks in generated text to track and verify AI-authored content across platforms.$Commercial implications: Enables selling watermarking tools to platforms needing content verification and authenticity tracking.
Authors
Yupeng Qiu, Han Fang, Ee-Chien Chang
Abstract
Large language model (LLM) watermarking operates on the next-token distribution that evolves during generation. In streaming generation, each sampled token becomes part of the text and affects subsequent distributions. Embedding therefore proceeds with a changing host, unlike classical watermarking, which adapts embedding to an existing host signal. To accommodate this difference, we formulate a novel embedding framework for LLM watermarking based on two principles. We first argue that embedding should be adaptive, retaining the state of previous sampling decisions to better guide subsequent choices. We further argue that embedding should be host-dependent, using the current LLM distribution to constrain both token selection and state updates. Guided by these principles and the two requirements of distribution preservation and robustness to text editing, we propose ClockMark. It combines adaptive and host-dependent embedding with message spreading, embedding the full payload in each watermarked token. Theoretically, we prove that ClockMark preserves the original LLM sequence distribution, thereby maintaining generation quality. Empirically, extensive experiments against 12 baselines demonstrate improved robustness to text editing, with gains of up to 16.08 percentage points in bit accuracy over the strongest tested baseline.