Self supervised learning improves with curated ocean sound data
Drowning in Noise: On Signal Saliency in Self-Supervised Learning for Ocean Acoustics
Sound
Summary
This paper looks at how computers learn to recognize sounds from the ocean. Normally, teaching computers this way works better with lots of data. But ocean recordings are mostly noisy sounds like waves and wind, with only rare animal noises. The authors show that if the computer is trained on carefully picked parts with animal sounds, it learns better and faster than using huge amounts of unfiltered noisy recordings. This means picking the right data is more important than just having lots of it for ocean sounds.
What this means in practice
- •For marine monitoring teams: Improve detection and classification of marine mammals by training models on carefully selected sound segments with animal vocalizations, reducing training time and data needs.
- •For environmental acoustic analysts: Enhance acoustic surveys by filtering ocean sound recordings to focus on biologically relevant segments before model training for better species identification.
Authors
Shikhar Bharadwaj, Samuele Cornell, Shinji Watanabe, Malcolm Slaney
Abstract
Self-supervised learning (SSL) for audio typically benefits from larger pre-training corpora, but this relationship assumes that training data carries meaningful learning signal. In ocean acoustics, this assumption is often violated: passive recordings are dominated by waves, wind, and silence, with biological vocalizations occupying a small fraction of the total duration. We show that this overrepresentation of noise causes SSL models to allocate representational capacity to acoustically trivial structure, degrading downstream performance. Using BEATs as a testbed, we demonstrate that pre-training on 30 hours of salient recordings (segments containing animal vocalizations) outperforms 1,240 hours of unfiltered data on a 31-class marine mammal classification benchmark. Under linear probing, the manually curated salient set achieves markedly better accuracy than the full corpus, despite using 41 times less data. Automatic saliency-based filtering improves over unfiltered pre-training, but still falls well short of the manually curated salient set. Our findings suggest that, in domains where informative content is temporally sparse, data curation matters more than data scale for SSL pre-training.