Language model fingerprinting improves text quality by rethinking watermark placement
Language Model Fingerprinting Requires Rethinking Watermark Teachers
Cryptography and SecurityArtificial Intelligence
Summary
Fingerprinting language models helps owners recognize their AI behind hidden services by embedding special watermarks. The authors found that old methods made the text worse while trying to keep the watermark strong. By changing where and how the watermark is placed, focusing on less obvious but still detectable parts of the text, they made the text sound better and kept fingerprints easy to spot. This approach lets verification gather clues over multiple uses instead of relying on one output, making detection more flexible and effective.
What this means in practice
- •For llm developers: Embed sparse, detection-friendly watermarks in models to better identify them without hurting text quality when users query APIs.
- •For content moderation teams: Improve detection of AI-generated text even when outputs vary, by using fingerprinting that aggregates signals from multiple queries.
Authors
Jeongyeon Hwang, Anshul Nasery, Sewoong Oh, Jungseul Ok
Abstract
LLM fingerprinting via watermark distillation embeds a statistical watermark signal into model weights, enabling model owners to identify their models behind black-box APIs. Revisiting a recent protocol, we find that its utility evaluation understates text quality degradation in open-ended generation, favoring overly strong watermark teachers. Weakening the watermark improves text quality but sacrifices detectability. To move beyond this trade-off, we rethink whether text watermarks designed for verifying generated text are suitable distillation teachers for model fingerprinting. Such watermarks are typically designed to remain detectable from an individual output, limiting how sparse the watermark signal can be. In contrast, fingerprint verification can aggregate signal across queries, making sparser watermark signals viable. This raises a key question: where should the sparse signal be placed? We analyze signal placement through token surprisal and show that, even at comparable watermark strength, different placements can target tokens with different plausibility under the base model. This motivates near-tie restriction, which uses top-1-relative logit gaps to restrict the watermark bias to tokens close to the base model's top prediction. Across multiple models, near-tie improves detection--quality frontiers under deployment changes, preserves higher text quality across query budgets, and further improves existing watermarking schemes when combined with them.