Audio watermarking method robust to neural codec resynthesis attacks
NeuMark: Neural Codec Resynthesis-Robust Audio Watermarking in the Codec Latent Space
Sound
Summary
Watermarks in audio files help trace the origin of generated speech, but new speech codecs can unintentionally erase these watermarks. The authors propose NeuMark, a method that hides watermarks inside the hidden data structures of speech codecs to survive such resynthesis. By distributing the watermark across aligned audio tokens, NeuMark improves the chances of both detecting the watermark and recovering its message after processing. This approach balances keeping the water unchanged to human ears and making the watermark strong enough to survive advanced speech processing.
What this means in practice
- •For speech technology developers: Embed watermarks into neural codec audio tokens to trace speech origins even after resynthesis with advanced speech codecs.
- •For digital audio forensic teams: Recover and detect watermarks from speech audio that has undergone neural codec resynthesis to verify authenticity.
Authors
Annan Wu, Wen-Chin Huang, Tomoki Toda
Abstract
Audio watermarking is increasingly important for tracing generated speech. Several audio watermarking methods have been proposed to embed the watermark in various domains, such as waveform, timbre feature, or latent representations, for making the embedded watermark robust against traditional digital signal processing (DSP) attacks. On the other hand, modern neural codecs introduce a different threat from DSP attacks: they resynthesize speech through quantized acoustic representations and can remove the embedded watermark evi- dence that is not aligned with codec-preserved structure. In this paper, we propose NeuMark, a codec-latent audio watermarking framework that embeds watermark evidence into SpeechTok- enizer acoustic tokens to address this resynthesis threat. NeuMark uses cross-attention to inject a 16-bit message across residual vector quantization (RVQ) layers, distributing the watermark over codec-aligned latent structure. Experimental results show that NeuMark substantially improves robustness under neural- codec resynthesis while supporting both watermark detection and message recovery. We also analyze the trade-off between reconstruction-referenced transparency and original-referenced robustness.