Audio emotion recognition adapted for people with sensory sensitivity
Audio emotion recognition for atypical hearing
Machine Learning
Summary
People with autism often hear sounds more strongly and differently, which makes understanding their feelings from sound tricky. The authors work on recognizing emotions in sounds for these individuals by using knowledge from typical listeners. They start by adjusting a big AI model to better predict emotions based on sounds using only a small set of example data. This helps make emotion recognition more personalized for people with sensory differences.
What this means in practice
- •For assistive technology developers: Improve emotion recognition components in hearing aids or sensory devices for users with auditory hypersensitivity.$Commercial implications: Enables new emotion-aware hearing aids tailored for autistic individuals with sensory sensitivities, improving user experience.
- •For mental health tool builders: Incorporate refined audio emotion recognition to monitor emotional states in people with atypical hearing for better mental health support.
Tested on one dataset.
Authors
Ulysse Roussel
Abstract
My doctoral work aims to explore Audio Emotion Recognition (AER) in the context of atypical listening. This research focuses on auditory hypersensitivity in people with autism, a phenomenon that is often difficult to evaluate and unique to each individual. Our core idea is to leverage our understanding of affect from acoustic traits, relying on the possibility of generalizing affective responses from a small amount of annotated data. As a first step, we fine-tune a large foundation model, Contrastive Language-Audio Pretraining (CLAP) using low-rank adaptation (LoRA), trained on a valence and arousal dataset of neurotypical listeners.