Audio fingerprint methods perform worse on speech than on music
Applicability and Generalization of Audio Fingerprinting Methods to Speech
SoundMultimedia
Summary
Audio fingerprinting helps recognize sounds by their unique patterns, even if they are changed or noisy. The authors studied how well these methods work with speech sounds, not just music. They found that newer self-learning approaches do better than older ones for speech, but still not as well as they do for music. When mistakes happen, the system is more likely to confuse voices of the same person than different words spoken by different people.
What this means in practice
- •For voice assistant developers: Improve speech match accuracy in noisy or altered audio environments using advanced fingerprinting models tested on speech data.
- •For broadcast monitoring teams: Detect and verify speech content from audio streams more reliably with fingerprinting methods that distinguish speakers better than text content.
Authors
Kemal Altwlkany, Elmedin Selmanovic, Sead Delalić, Emanuel Lacic
Abstract
Audio fingerprinting methods identify audio based on its content, even when the content has been possibly corrupted. Despite their applicability and usage in the speech domain, research and advancements in audio fingerprinting traditionally focus on music and are almost exclusively evaluated on music. We provide insights into speech-oriented audio fingerprinting by benchmarking the best performing audio fingerprinting models on speech data while taking into consideration the effects of noise, reverberation, temporal changes, pitch modifications, and lossy compression. Our findings indicate that audio fingerprinting methods based on self-supervised learning outperform classical approaches on speech data, while the performance of these models on speech is inferior compared to the performance that they achieve on music. Results also include an analysis of false positives, showing that fingerprinting methods are more likely to match with an audio of the same speaker rather than an audio of the same text, when mismatches occur.