WavLM layers reveal how speech models detect fake audio attacks

Interpreting SSL Representations for Spoof Detection: a WavLM Study

SoundMachine Learning

Summary

Detecting fake or spoofed speech is important for security, but it's unclear how self-supervised speech models like WavLM do it. The authors studied which layers of WavLM best detect various fake speech attacks and what sound features they focus on. They found mid layers capture general audio traits while later layers detect signs related to voice quality. These deeper layers don’t just spot specific attacks but recognize subtle artifacts common across different types of fakes.

What this means in practice

  • For voice security teams: Improve spoof detection systems by focusing on specific WavLM layers that capture key acoustic artifacts of fake audio.
  • For audio forensics analysts: Use insights about which speech features correspond to different WavLM layers to better analyze synthesized or manipulated audio evidence.

Tested on one dataset.

Authors

Mohamed Mallat, Michele Panariello, Massimiliano Todisco, Nicholas Evans, Anthony Larcher

Abstract

Self-supervised speech models such as WavLM achieve strong spoofing detection performance, yet which layers encode the most spoof-relevant information and what acoustic properties they capture remain opaque. We present a layer-wise interpretability study of WavLM using the ASVspoof 5 database, combining three complementary analyses: (i) per-layer linear probing with attack-level evaluation across speech synthesis, voice conversion, and adversarial attacks; (ii) a canonical correlation analysis (CCA) linking layer representations to spoofing-relevant acoustic features; (iii) an exhaustive grid search over layer subsets showing that use of five layers outperforms full thirteen-layer pooling. CCA analysis reveals that, while most acoustic features peak in alignment with mid-layers and then decay, voice quality measures jitter, shimmer, and the harmonics-to-noise ratio show the opposite trend, with correlation increasing toward later layers. Embedding visualisations reveal that deeper layers capture a continuous acoustic similarity among attacks rather than discrete categories, suggesting these layers encode fundamental synthesis artifacts rather than attack-specific signatures.