Speech emotion recognition improves with conversational context
Enriching Speech Emotion Representations with Conversational Context
Computation and Language
Summary
Understanding how people feel in conversations can help make speech-based systems smarter and friendlier. The paper introduces a new method called ACERT that looks not just at single sentences but at several nearby parts of a conversation to better understand emotions. This method was tested on different types of emotional speech and worked better than previous methods. The improvement comes from tracking how emotions change over time during a conversation, rather than just from who is speaking or the sound quality.
What this means in practice
- •For voice assistant developers: Incorporate conversation context windows to improve emotional understanding in spoken AI assistants for more natural interactions.$Commercial implications: Enables smarter AI voice assistants that detect evolving emotions, enhancing user engagement and satisfaction.
- •For call center technology teams: Use context-aware emotion recognition to monitor customer sentiment more accurately over the course of calls.
Authors
Arthur Peuvot, Romaric Besançon, Gaël de Chalendar, Bianca Vieru, Ioana Vasilescu
Abstract
Detecting emotions is necessary for building systems that can accurately and adaptively interact with humans. Speech Emotion Recognition (SER) has become an important research focus to develop intelligent spoken interfaces. However, most studies predict emotions at the utterance level, ignoring the conversational context, along with the emotional flow and speaker interactions it carries. In this paper, we introduce ACERT (Averaged Contextual Emotion Representation through Time), a module that integrates a flexible-length window of conversational context to better capture emotional evolution in spoken interactions. To evaluate the robustness of this method, we conducted experiments on datasets spanning diverse emotionally expressive styles and contexts. ACERT outperforms current state-of-the-art (SOTA) approaches on IEMOCAP, establishes the first context-aware benchmark on SAFE, and obtains strong results on MELD for unweighted, class-balanced metrics. Ablation studies show that ACERT's gains come from emotional and conversational continuity, rather than from speaker identity or acoustic conditions.