Deep learning models compared for heart signal emotion recognition
Bridging the Gap in ECG-Based Emotion Recognition: A Unified Evaluation of Deep Learning Models
Machine Learning
Summary
It can be hard to tell which computer methods best recognize human feelings from heart signals because studies use different data and tests. The authors combined several heart signal datasets into one to create more varied data. They tested different deep learning models using this combined data to see which work best across different conditions. They also created open tools to help others do similar fair comparisons. This work helps make emotion recognition from heart data more consistent and reliable.
What this means in practice
- •For wearable device engineers: Develop ECG-based emotion recognition features that perform consistently across varied sensors and user groups using the standardized benchmark and datasets.
- •For mental health app developers: Integrate reliable emotion detection from ECG signals that generalize well beyond specific datasets, improving real-world user experience.
Authors
Timothy C Sweeney-Fanelli, Ajan Ahmed, Masudul Imtiaz
Abstract
Deep learning has led to numerous proposed architectures for Automated Emotion Recognition (AER) from electrocardiogram (ECG) data, but inconsistencies in preprocessing, training, and evaluation make direct comparisons difficult. Most studies train and validate models on individual datasets collected under homogeneous conditions, limiting variability and raising concerns about generalizability. Cross-dataset validation is sometimes used but primarily assesses model adaptability rather than true generalization. This study presents a comparative analysis of prominent deep learning architectures in AER, emphasizing model generalization over dataset adaptability. To enable this benchmark, we introduce two open-source frameworks: Affective Research on Representations and Classifications (ARRC), a standardized benchmarking toolkit, and Affective Research Dataset Toolkit (ARDT), a framework for inter-dataset training and validation. Using ARDT, we consolidate three publicly available AER datasets, CUADS, ASCERTAIN, and DREAMER, into a single dataset, increasing variability in sensor types, recording conditions, and participant demographics. We then use ARRC to evaluate three widely studied deep learning models and two CNN baselines through hyperparameter optimization and 10-fold cross-validation. Our findings provide insights into the trade-offs between classification accuracy and model complexity, establishing a reproducible benchmark for AER research. All source code for ARRC, ARDT, and model evaluation is publicly available to ensure transparency and facilitate further research.