Finetuning Strategies for Querying Sounds by Vocal Imitation
2026-08-19 • Sound
SoundArtificial IntelligenceInformation Retrieval
AI summaryⓘ
The authors describe their winning entry in a challenge where the goal was to find sound effects based on someone imitating them with their voice. They tried two main methods: one used a special sound-processing tool that was kept fixed, while learning to compare sounds, and the other combined two learning techniques with a different sound recognition model. They updated their report to add more detailed information released after the competition. Overall, their work focuses on improving how machines understand vocal imitations to find matching sound effects.
contrastive learningtriplet learningfrozen encoderCED encoderMobileNetV3vocal imitationsound effects retrievalsemi-hard negativesAES AIMLA challenge
Authors
Aditya Bhattacharjee, Christos Plachouras, Sungkyun Chang, Emmanouil Benetos
Abstract
This technical report describes our winning submission to the AES AIMLA 2025 Challenge on querying sound effects by vocal imitation. We investigate two complementary fine-tuning strategies: contrastive learning with a frozen, pretrained CED encoder, and joint contrastive-triplet learning with semi-hard negatives using a MobileNetV3 encoder. This report has been updated for posterity to include details released after the challenge.