Large language models struggle with understanding emotional relations in group talks
RESCUE-BENCH: Towards Relation-Aware Multi-Party Emotional Support Conversation Systems
Artificial Intelligence
Summary
Most systems that offer emotional support focus on one person talking to one helper, but real-life conversations often involve families or couples with complex relationships. This paper introduces a new way to test if AI models can understand these relationships and give better emotional help. The authors made a dataset of real conversations with couples and families, marking emotional and relationship details in each part. They found that current AI models can catch simple emotional signs but have trouble understanding and using the deeper relationship dynamics to provide sensitive support.
What this means in practice
- •For mental health app developers: Develop chatbots that better understand family or couple dynamics to provide more empathetic emotional support in multi-person conversations.
- •For customer service teams: Improve AI tools to recognize and address complex emotional situations involving multiple participants, such as family disputes or group service interactions.
Authors
Haichuan Hu, Yang Xiao, Mingni Tang, Jiawen Duan, Quanjun Zhang, Congqing He, Hao Zhang, Jiashuo Wang, Johan F. Hoorn, Wenjie Li
Abstract
Existing emotional support conversation systems mainly focus on one-on-one seeker-supporter interactions and individual emotional states, leaving interpersonal relations in multi-party scenarios underexplored. In this work, we introduce relation-aware emotional support conversation, a new task that evaluates whether LLMs can capture and utilize the evolving dynamics of relationships to offer more effective emotional support. We construct RESCUE (Relation-aware Emotional Support Conversation Understanding and Evaluation Benchmark) from real couple and family interview conversations, containing 191 samples, 7,079 annotated turns, and 1,064.8 minutes of video. Based on rich annotations of socio-emotional and support-related dynamics, RESCUE defines six tasks that evaluate two core capabilities required for relation-aware emotional support: Relational Understanding and Relation-Sensitive Support. Experiments with ten LLMs show that current models perform relatively well on tasks relying on local emotional or intervention cues, but struggle with relation-intensive tasks such as relation pattern prediction, viewpoint prediction, and support strategy prediction. These findings reveal the limitations of current LLMs in modeling interpersonal relations and making relation-sensitive support decisions.