AI empathy shapes user privacy attitudes and sensitive sharing
How Does Perceived Empathy From AI Chatbots Shape Users' Privacy Attitudes, Behaviors, and Needs?
Human-Computer Interaction
Summary
People share more personal information with AI chatbots when the bots seem to understand and help them, even if the bots don’t feel real empathy like humans do. The study found that users think real empathy includes caring and resonating, but AI mainly shows understanding and helpfulness. This can still lead to privacy risks because users may disclose sensitive details. The researchers also found that users want different levels and kinds of empathy from AI, which affect their privacy concerns.
What this means in practice
- •For chatbot developers: Design privacy features in chatbots tuned to varying empathy levels to protect sensitive user information during interactions.
- •For privacy compliance teams: Create guidelines to manage user privacy risks related to sensitive disclosures driven by chatbot empathy capabilities.
Authors
Jiaxun Cao, Hiba Laabadli, Yuntong Xue, Jana Schaich Borg, Pardis Emami-Naeini
Abstract
Prior research suggests that some dimensions of AI empathy may increase users' disclosure to chatbots, posing growing privacy risks as chatbots become more capable of empathic interaction. Yet this work often conceptualizes AI empathy narrowly, leaving users' own definitions and their broader privacy experiences unexplored. We interviewed 21 AI Chatbot users about what AI empathy meant to them and how it shaped their privacy attitudes, behaviors, and needs. Participants characterized empathy through five components--understanding, resonating, caring, expressing, and helping. Most participants viewed resonating and caring as essential components of human empathy but absent from AI empathy. Yet AI's contextual, in-depth understanding and helpfulness still encouraged highly sensitive disclosure. This highlights that AI empathy can endanger users' privacy without necessarily making AI seem human. Participants also desired varied forms and levels of empathy, each associated with distinct privacy needs. We offer actionable recommendations for designing privacy-enhancing mechanisms for empathic AI interactions.