OpenStamp: A Watermark for Open-Source Language Models
Computation and LanguageArtificial IntelligenceMachine Learning
Summary
The authors created OpenStamp, a new way to mark texts generated by open-source language models to prove they came from the model. Unlike earlier methods that can be easily turned off, OpenStamp hides the mark inside the model's final layer, making it harder to remove. Their tests show OpenStamp works better and keeps the model's abilities intact. It also resists attempts to change the text or retrain the model to erase the mark. The authors shared their code and marked versions of popular models for others to use.
Authors
Miroojin Bakshi, Saksham Rastogi, Danish Pruthi
Abstract
With the growing prevalence of large language model (LLM) generated content, watermarking is considered a promising approach for attributing text to LLMs and distinguishing it from human-written content. A prominent class of techniques embeds subtle but detectable signals in generated text by modifying token sampling probabilities. However, such methods are unsuitable for open-source models, where users have white-box access and can easily disable watermarking during inference. In this work, we introduce OpenStamp, a watermarking technique that encodes the watermarking logic directly into the model weights by modifying only the final projection, or unembedding, layer. Through experiments across two models, we show that OpenStamp achieves superior detection performance, with minimal degradation in model capabilities compared to prior methods. The implanted watermark is explicitly designed, and empirically confirmed, to be more robust to paraphrasing attacks and harder to scrub off through post-hoc fine-tuning than prior open-source watermarks. To enable developers to watermark their models, we release our code alongside watermarked versions of 4 popular open-source models.