Text-to-speech model embeds robust audio watermarks in neural codec layers
NeuMark-Native: Robust Text-to-Speech-Native Watermarking Through Full Utilization of Neural Audio Codec Latent Space
Sound
Summary
Creating fake or synthetic speech can raise concerns about where the audio came from and who made it. Most existing methods add hidden watermarks to the finished audio, but these can be removed or weakened during processing. The authors propose a new way to embed invisible watermarks directly inside the internal language of the neural speech generator before the sound is produced. This makes the watermark harder to remove even after complex audio changes while keeping the speech natural and clear.
What this means in practice
- •For voice assistant developers: Embed persistent watermarks within synthetic speech to verify source authenticity even after signal modifications.
- •For content distribution platforms: Detect and trace synthetic audio origins robustly despite multiple compression or processing stages for copyright protection.
Authors
Annan Wu, Wen-Chin Huang, Tomoki Toda
Abstract
Speech watermarking offers proactive traceability for synthetic speech, yet most existing models operate only after text-to-speech (TTS) synthesis by adding a watermark perturbation to the generated waveform. This post-hoc design leaves watermarking as an external step that can be omitted or bypassed and restricts the watermark to a shallow waveform representation. We propose NeuMark-Native, a TTS-native watermarking framework for neural codec-based synthesis. It embeds payload information into every generated codec-latent layer before waveform decoding, improving watermark persistence under downstream digital signal processing (DSP) and neural codec resynthesis. NeuMark-Native keeps the pretrained TTS model and the neural codec frozen, while optimizing only the watermark modules on generated codec tokens. Experiments on two corpora under 11 DSP attacks and 9 neural-codec attacks demonstrate robust watermark detection while preserving naturalness, intelligibility, and speech quality close to synthetic speech.