Hidden in Plain Tokens: Simply Robust, Gradient-Free Watermark for Synthetic Audio
2026-05-25 • Machine Learning
Machine LearningSound
AI summaryⓘ
The authors address the challenge of watermarking synthetic audio generated by AI, which is tricky because sounds are continuous and often converted into discrete tokens. They found that errors in token detection hurt watermark reliability and solved this by shrinking the set of tokens using a method called community detection. This approach strengthens watermark detection without needing to retrain the AI, making it more robust and practical. Their work sets a new standard for watermarking in multimedia by focusing on how discrete audio representations behave.
generative AIwatermarkingsynthetic audioautoregressive modelsdiscretizationtokenizercommunity detectiondiscrete representation learninggradient-free methodcontent provenance
Authors
Georgios Milis, Yubin Qin, Yihan Wu, Heng Huang
Abstract
As policy catches up with the capabilities of generative AI, watermarking is central to content provenance efforts. Inference-time watermarks for autoregressive models are unfit for continuous modalities due to discretization inconsistencies. Existing methods overcome this by finetuning the modality tokenizers, nullifying the watermark's training-free advantage. In this work, motivated by the vocabulary redundancy of discretization, we propose an elegant solution for powerful and robust watermarking of synthetic audio. We theoretically analyze the impact of token errors on watermark detection, and effectively mitigate them using a reduced vocabulary obtained via community detection. Thorough experiments showcase that our gradient-free method can boost detectability by several orders of magnitude, while also achieving built-in robustness to audio modifications. Broadly, we discover a new state-of-the-art for token-level watermarks in multimedia, which simply arises from the nature of discrete representation learning.