Hybrid AI music detection maps synthetic stems in mixtures
A Stem-Agnostic Approach to Hybrid AI Music Detection
Sound
Summary
Detecting AI-generated parts in music is harder when real and synthetic sounds are mixed together. The authors created a new way to find AI-made sections in complex music tracks by analyzing sounds across time and frequency. Their method works well for vocals, drums, and guitars but has trouble with bass sounds. They found that how well the music is separated into parts affects detection, highlighting a key challenge for future work.
What this means in practice
- •For music platforms: Identify AI-generated stems within uploaded music tracks to enable content authenticity labeling and rights management.$Commercial implications: Supports services that verify music originality for streaming platforms and copyright holders, enabling new content policing products.
- •For audio software developers: Incorporate stem-agnostic AI detection into audio editing tools to flag synthetic content during mixing and mastering.
Authors
Richa Namballa, François Rigaud, Romain Hennequin
Abstract
The inclusion of generative audio in the music production process has led to an increase in hybrid music tracks that blend authentic human performances with AI-generated stems, challenging traditional AI music detectors which operate in a binary setting. In this work, we propose a stem-agnostic framework for identifying synthetic audio sources within hybrid musical mixtures. We introduce the inspectrogram, a novel time-frequency representation that maps localized probabilities of synthetic content across the audio spectrum. By combining the inspectrogram with a Wiener filter estimating target stem energy dominance, a single CNN model evaluates whether the specific stem is generated. Trained on rendered hybrid mixtures and evaluated across various stem classes, our model achieves strong performance on high-frequency sources such as vocals, drums, and guitar, but struggles on the low-frequency, narrow-band bass. We conclude that the quality of separation impacts the detection accuracy and identify source separation as a primary bottleneck and a crucial direction for future research.