Audio model accuracy depends on event class changes in sound clips
Beyond Answer Changes: Matched Same-Class Controls for Local Audio Evidence
SoundArtificial Intelligence
Summary
This paper looks at how audio-language models respond when small parts of a sound clip are changed. The authors found that when a replaced sound belongs to a different class, the models' answers change a lot more than when replaced with a sound from the same class. This means the models are sensitive to what kind of sound event is present, not just any waveform change. Their tests help measure how well these models focus on specific sound events when answering questions.
What this means in practice
- •For audio engineers: Evaluate sound recognition models’ sensitivity to local changes in audio event classes using matched replacements.
- •For audio model developers: Design tests that pinpoint whether audio models rely on event class information rather than waveform details.
Authors
Yunuo Wang, Maoyuan Chen, Yuanjin Huang, Xing Liu
Abstract
For audio-language models, an answer change after a local sound-event edit can reflect a response to the event's class, sensitivity to waveform rewriting, or both. We compare matched same-class and different-class replacements of a queried event. Both use the same interval, level estimate, donor fitting, and crossfade, while the same-class replacement preserves the correct answer. Across 600 synthetic soundscapes and three models from two architecture families, different-class replacements produce 49.7 to 60.9 percentage points more answer changes than same-class replacements. Same-class accuracy remains within 5.2 percentage points of original accuracy. Among trials with correct original and same-class answers, soundscape-weighted selective accuracy for the inserted class is 73.7 to 81.9%, averaging eligible trials within each soundscape and then weighting eligible soundscapes equally. This protocol tests how model answers respond to local changes in event class.