Model activations reveal artist identity from song lyrics
Ghost in the Encoder: Decodable Artist Identity Representations in Lyrics-to-Song Generation
SoundMachine Learning
Summary
Text-to-song AI models can sometimes mimic specific singers or recreate whole songs they were trained on. The paper shows that these models keep secret clues about which artist a set of lyrics belongs to inside their internal workings, even without being told directly. This means the model learns to identify artists just from lyrics, and this information guides the way it creates songs. Understanding this hidden information helps audit what AI music tools really learn.
What this means in practice
- •For music app developers: Detect when song generation models implicitly mimic specific artists based on lyrics, to improve content originality safeguards.
- •For music rights teams: Audit AI-generated songs to identify unintended copying of artist styles hidden in model activations from lyrics inputs.
Authors
Arhan Vohra, Choenden Kyirong, Laura Ibáñez-Martínez, Martín Rocamora
Abstract
Text-to-song generation models can be prompted to imitate specific artists or regurgitate entire songs from their training data. Although these phenomena have been documented behaviorally on small datasets, little is known about the internal representations that may give rise to them. Prior interpretability work on generative audio has focused on locating semantic concepts such as genre or time signature within model activations. In this work, we show that a trained model can be probed for linearly decodable representations of artist identity from song lyrics alone, without any additional identifiers. Through a controlled case study of ACE-Step 1.5 spanning 2,000 songs across 100 artists, we demonstrate that the artist associated with a given set of lyrics can be identified within the model's internal activations, and that this conditioning signal propagates from the lyric encoder to the diffusion backbone during inference. These findings indicate that lyrics constitute an artist-level conditioning channel not addressed by prompt-side replication safeguards. More broadly, our work highlights how latent-space analysis can be used to audit what generative music models have implicitly learned from their training data.