How ai companions shape user perceptions in human ai relationships
The AI Side of the Human-AI Companion Relationship: Contributions to Users' Mental Models
Human-Computer Interaction
Summary
People often feel like their AI companion truly understands and responds to them like a friend would. The authors explain that while users think the AI changes and grows like a real person, the AI actually doesn’t have a continuous self or deep personal change. Instead, the AI’s adjustments are external and can be reset or edited. This difference means users may wrongly believe their AI is more like a real partner than it really is, which can affect how they think about their relationship with it.
What this means in practice
- •For ai product designers: Design AI companions with clearer communication about how AI responses are generated to prevent users from overestimating interpersonal significance.
- •For mental health app developers: Develop AI companion features that set appropriate user expectations to avoid misunderstandings about AI empathy and adaptation.
A position paper. It proposes an approach and reports no results.
Authors
Richard A. Fabes, Carol Lynn Martin, Gautam Dasarathy, Vignesh Tirukkonda
Abstract
For many users, the relationship they have with an AI companion (AIC) feels genuinely mutual. The AIC appears to be attentive, empathic, and responsive, and the exchange can feel like two parties reacting to each other in real time. As a result, the user's mental model of relational adaptation correctly represents the AIC's responsiveness and change, but the mechanisms producing this may not be what the user thinks they are. In this paper, we illustrate how an AIC's apparent mutual engagement is generated without a persisting relational subject on the other side and without a self that persists between prompts. Despite this, we show that both the human and the AIC are changed by the interaction, but these changes occur at different levels. The change in the person is constitutive (i.e., intrinsic), but the change in the AIC is configurative (i.e., external and editable), and it is this difference in level that explains how the AIC contributes to a sense of mutuality. Importantly, we highlight how the mental models users develop regarding AIC mutuality may misrepresent its mechanisms and ontological significance. Specifically, we discuss how the ways in which AICs respond to a user contribute to the user's inaccurate inference that the AIC's adaptation is constitutive rather than configurative and how this can lead users to assign unwarranted interpersonal significance to the mutuality of the relationship. The paper specifies the implications this has for users, researchers, and AIC designers.