CD-MED: Cross-Domain Multimodal Emotion Descriptor for Visual Comparison of Digital Objects
2026-07-14 • Human-Computer Interaction
Human-Computer Interaction
AI summaryⓘ
The authors created CD-MED, a tool that helps understand emotions from different types of digital content like movies, songs, and books by putting all their emotional info into one common format. Each type of content is analyzed with its own method, and then the results are combined into a shared emotional descriptor. This descriptor shows emotions visually by using position, color, size, and shape, making it easier to compare and explore emotions across different media. Their approach helps in tasks like searching and recommending content based on feelings.
emotion recognitionmultimodal analysisvalence-arousal spacecross-domain representationdigital mediaaffective computingemotion descriptormodalityemotion visualizationemotion retrieval
Authors
Elnara Kadyrgali, Muragul Muratbekova, Pakizar Shamoi
Abstract
Digital objects express emotions through different modalities. For example, a movie may include visual scenes, audio, dialogue, and facial expressions, while a song may contain melody, rhythm, lyrics, and vocal tone. Because existing emotion recognition models are usually modality-specific, it is difficult to compare such objects directly. This paper proposes CD-MED, a Cross-Domain Multimodal Emotion Descriptor for representing heterogeneous digital objects in a common emotional space. Each modality can be processed by its own emotion recognition model, and the resulting emotional outputs are transformed into a shared descriptor. The descriptor preserves information from individual modalities while also allowing an integrated emotional profile of the object. For interpretation, CD-MED is visualized in the valence-arousal space: position represents affective coordinates, color denotes emotion category, size indicates intensity, and shape shows the modality. This unified representation enables emotion-based comparison, retrieval, recommendation, and visualization across different domains such as movies, songs, images, and books.