Reinforcement learning improves emotional reasoning with diverse and grounded predictions
DSPO: Diversity-aware Subjective Policy Optimization for Robust Emotional Reasoning
Computer Vision and Pattern Recognition
Summary
Understanding human emotions from images is very hard for AI because feelings can be different for everyone and are not directly visible. The authors found that current AI methods struggle because they treat emotions as fixed labels and do not consider the variety of feelings people might have. They propose a new learning approach that encourages the AI to suggest a diverse set of emotions that better match the context of the image, while also checking that these feelings are supported by the actual visual clues. Their method improves the AI's ability to guess emotions more accurately, especially when tested on different kinds of images.
What this means in practice
- •For multimodal ai developers: Enable improved emotional reasoning in AI that interprets images with emotionally diverse and visually grounded responses.
- •For affective computing engineers: Enhance systems that detect and respond to human emotions from visual data by using diverse subjective emotional predictions tied to visual evidence.
Authors
Cheng Ye, Weidong Chen, Bingyan Xu, Zhendong Mao
Abstract
Reinforcement Learning has significantly advanced the complex reasoning capabilities of MLLMs. However, prevailing RL algorithms suffer a severe failure in emotion reasoning tasks. These methods heavily rely on deterministic hard-label supervision and point-wise isolated evaluation, creating a fundamental gap with the inherently subjective and continuously distributed nature of human emotions. Furthermore, unlike explicit physical objects, emotional states are deeply implicit within visual cues. This abstract nature exacerbates visual hallucinations in MLLMs, leading to plausible yet ungrounded emotional evidence. To address these limitations, we propose Diversity-Aware Subjective Policy Optimization (DSPO), a reinforcement learning framework that jointly promotes subjective affective coverage and visual grounding. First, we construct a context-grounded emotional distribution prior in the VAD space by combining the lexical prior of the annotated emotion with image-specific contextual information. Based on this prior, we introduce a Distribution-Aligned Emotional Diversity Reward (DEDR), which measures the leave-one-out marginal contribution of each candidate emotion within a rollout. DEDR rewards candidates whose inclusion brings the predicted affective set closer to the context-grounded prior, thereby preserving plausible subjective interpretations without encouraging unconstrained dispersion. We further develop Counterfactual Visual Intervention Gating (CVIG), which masks the visual region highlighted in the reasoning process and uses the resulting candidate-wise probability changes to reduce the weights of interpretations unsupported by visual evidence. Extensive experiments demonstrate that DSPO achieves state-of-the-art performance across multiple public benchmarks, especially on the cross-domain performance, i.e., improving +10.8\% on average cross-domain accuracy than EMO-R3.