Improving detection and extraction of sounds in audio scenes
Evaluating Inference-time Algorithms for Semantic Sound Scene Segmentation
Sound
Summary
Detecting and pulling out specific sounds in a complex audio scene is tricky because mistakes in finding sounds can mess up the process of isolating them. The authors created a model using multiple audio channels and tested different ways to improve accuracy when identifying and extracting sounds. They compared various methods that decide when and how to update their guesses during listening. Their work shows how to better fix errors during the listening step, which could help make sound analysis more reliable in real-world situations.
What this means in practice
- •For audio engineers: Improve sound event detection systems to better separate target sounds from complex audio environments.
- •For smart device developers: Enhance voice assistant or surveillance devices by reducing errors in sound recognition during live audio processing.$Commercial implications: Enables more reliable real-time sound recognition products for consumer devices by refining inference-time updates.
Tested on one dataset.
Authors
Sripathi Sridhar, Gordon Wichern, Yoshiki Masuyama, Mark Cartwright, Jonathan Le Roux
Abstract
Spatial semantic segmentation of sound scenes (S5) involves the detection and extraction of target sound events in audio files. However, in typical detection-extraction pipelines, errors in the detection module propagate to the extraction module, degrading overall performance. In this work, we develop a multichannel detection-extraction model and evaluate inference-time algorithms to improve the class detection and target sound extraction in a multi-stage setup. We evaluate two fitness functions: binary cross entropy and mixture consistency; two relabeling strategies: naive and conditional confusion matrix relabeling; and three source estimate evaluation models: a multi-label classifier, a fine-tuned single-source classifier, and an off-the-shelf audio judge. We thoroughly evaluate the impact of these design choices on the DCASE 2025 Task 4 dataset, and validate entropy-based gating of inference-time updates on the held-out evaluation set, laying the groundwork for inference-time updates of S5 systems in real-world scenarios.