Ai chatbots often misgender users causing identity frustrations
Misgendering as Breakdown in Human-Machine Communication: How AI Companion Chatbot Users Experience and Repair Misgendering
Human-Computer InteractionComputers and Society
Summary
Sometimes AI chatbots call people by the wrong gender, which can make users feel misunderstood or upset. The authors looked at many online posts where users talked about being misgendered by these chatbots. They found users try different ways to teach the chatbot the right gender or fix mistakes. The paper also talks about how developers can make chatbots better at recognizing and respecting gender identity.
What this means in practice
- •For ai chatbot developers: Design chatbot interactions that reduce misgendering and improve user gender presentation recognition.
- •For social media platform teams: Moderate and support online communities discussing AI chatbot identity challenges to foster safe digital role play spaces.
Tested on one dataset.
Authors
Julia Liu, Qing Xiao, Leona Yinglang Pang, Haiyi Zhu, Hong Shen, Jordan Taylor
Abstract
In recent years, large language model-based AI companion and role play chatbots have grown increasingly popular. People turn to these chatbots for emotional support and to engage in romantic and erotic role play. Although prior research suggests that digital role play can help people explore their gender and sexuality, LLM based technologies are also replete with gender and sexuality biases. In this study, we examine one way that AI chatbots can harm users: misgendering. In order to study chatbot misgendering we qualitatively analyzed 326 posts mentioning misgendering that were shared in AI companion or role play subreddits. We document how chatbot misgendering takes place and how, in response, users engage in ongoing work to curate their gender presentation to prevent and repair misgendering. We discuss how researchers and designers can mitigate chatbot misgendering and consider the implications of using AI chatbots for identity exploration.