Conversational agent boosts trust by asking questions not giving advice
Trust by Design: Trust Calibration Through Non-Advisory Socratic Dialogue in Conversational Agents
Software EngineeringComputers and SocietyHuman-Computer InteractionMultiagent Systems
Summary
When people use AI to get help, sometimes they trust it too much or too little, especially in serious situations. The authors made a chatbot called CASELy that never gives advice but asks questions based only on what the user says. This way, people stay in control and understand the AI is just helping them think, not telling them what to do. In a study, students liked this style of interaction more, feeling it was clearer and less judgmental.
What this means in practice
- •For customer support teams: Use CASELy’s questioning method to reduce customer overreliance on AI advice in sensitive support contexts.
- •For mental health app developers: Incorporate non-advisory dialogue to encourage users to reflect rather than depend on AI guidance in mental wellness tools.
Authors
Roba Hassan, Nahla Aboromi, Naomi Unkelos-Shpigel
Abstract
As conversational AI systems increasingly operate in sensitive domains, the central challenge shifts from usability to trust calibration, ensuring that users rely on systems neither too much nor too little. Systems that provide advice or interpretations risk encouraging inappropriate reliance, particularly when users perceive AI outputs as authoritative. We present CASELy, a conversational agent explicitly designed to limit its own authority through non-advisory Socratic dialogue. The agent asks reflective questions grounded exclusively in user input and refuses to provide advice, recommendations, or interpretations. This design operationalizes trust calibration by constraining agent agency rather than optimizing capability. In a pilot randomized controlled study with higher education students, participants interacting with the Socratic dialogue reported substantially higher user experience (UEQ-S overall = 1.50) compared to a non-dialogue control (0). Qualitative findings identify three mechanisms supporting calibrated trust: transparency through visible grounding, preservation of user decision authority, and reduced fear of judgment. We argue that appropriate reliance can be achieved through interactional constraints, offering a design pattern for trustworthy conversational AI in sensitive contexts.