Singing emotions linked to vocal techniques and sound features
Interpretable Singing Emotion Recognition Using Acoustic and Vocal Technique Features
SoundArtificial Intelligence
Summary
Recognizing emotions in singing usually depends on complex AI models that don't explain how singers express feelings. This paper shows a way to identify emotions using specific features of singing, like pitch changes, voice quality, and singing style. The authors also use tools to understand which singing traits help detect emotions and find that techniques such as vibrato and throat adjustments are important emotional cues. Their approach works well across different singers and helps explain the connection between how singing is done and the emotions expressed.
What this means in practice
- •For music therapists: Assess singers' emotional delivery using interpretable vocal technique features to tailor therapeutic interventions.
- •For voice coaching professionals: Provide singers feedback on technical aspects tied to emotional expression to enhance performance quality.
- •For speech technology developers: Incorporate singing-specific vocal features to improve emotion recognition models for singing voice applications.$Commercial implications: Enables development of emotionally aware singing analysis or tutoring products for commercial markets.
Authors
Fotini Kiriakopoulou, Vassilis Lyberatos, Spyridon Kantarelis, Athanasios Voulodimos, Giorgos Stamou
Abstract
Singing emotion recognition relies predominantly on deep learning models that achieve strong performance but offer limited insight into the vocal characteristics driving emotion, of- ten overlooking the singing voice as a distinct signal. We propose an interpretable singing emotion recognition framework using a compact set of singing-specific descriptors across prosody, spec- tral features, phonetics, and vocal technique. To decode model decisions and link predictions to measurable acoustic phenomena, we employ SHAP and apply GRAD-CAM to qualitatively assess the learned representations of vocal techniques. Evaluations show that singing-specific features reduce cross-singer variability, while vocal technique descriptors provide interpretable insights into singing expression. Furthermore, out-of-domain evaluation on VocalSet reveals interesting associations between emotional expression and ground-truth technical execution. The analysis highlights vibrato and pharyngeal techniques as salient cues of emotional delivery, with SHAP indicating that vibrato is context-dependent on articulation rate. These findings emphasize the value of domain-specific features for interpretable singing emotion analysis.