Masked audio pretraining improves sound detection for biodiversity monitoring
MAST: Label-Efficient, Robust, and Generalizable Sound Detection for Biodiversity Monitoring via Masked Audio Pretraining and Self-Training
SoundMachine Learning
Summary
Monitoring wildlife by listening to their sounds is hard because labeling specific animal calls in recordings takes a lot of work and doesn’t easily transfer across different places. The authors developed a new method called MAST that teaches a computer to recognize animal sounds with fewer labeled examples by first learning from lots of unlabeled recordings. Then it improves itself by guessing new labels from unlabeled data and learning from its own predictions. This approach works well in different environments, helping track animals with less manual effort.
What this means in practice
- •For environmental monitoring teams: Improve automated identification of animal sounds in diverse habitats with less labeled data for large-scale biodiversity surveys.
- •For wildlife conservation organizations: Deploy more robust and generalizable audio-based species detection systems that adapt to new sites without extensive retraining.
Authors
Tianyi Xu, Daniel Pimentel-Alarcón, Zuzana Buřivalová, Claudia Solís-Lemus
Abstract
Passive acoustic monitoring can measure biodiversity at larger scales, but time--frequency annotation of animal vocalizations is expensive, site-specific, and difficult to sustain at scale. We present a label-efficient sound detection framework that combines masked audio pretraining with a lightweight detector on mel spectrograms, then further improves robustness through iterative self-training on unlabeled audio. We first pretrain a ViT-based encoder on unlabeled recordings via masked reconstruction and transfer the encoder to a detection backbone. To better separate animal sounds from confounding background, we add a box-level contrastive loss that pulls matched event regions together while pushing noisy negatives apart. We then apply a two-stage pseudo-labeling curriculum to exploit large unlabeled pools without additional annotation. We evaluate the performance on two ecologically distinct domains: tropical rainforest soundscapes (Indonesia) and bird vocalizations in Mediterranean habitats (Spain). On both domains, masked audio pretraining and contrastive learning consistently improve time--frequency detection under temporal and cross-site distribution shift, and self-training yields further gains in out-of-distribution performance. On the rainforest domain, MAST with self-training achieves +0.22 mAP and +0.24 F1 over the strongest baseline under cross-site shift. On the bird domain, self-training achieves +0.12 mAP and +0.10 F1 over the strongest baseline under cross-site shift. Overall, our results show that MAST can effectively extend self-supervised audio representations from clip-level tasks to robust box-level localization across diverse bioacoustic settings, providing a practical path for biodiversity monitoring with limited labels.