People increasingly trust large language models for personal advice
LLMs as Oracles: Reliance on LLMs for Subjective Personal Questions
Computers and SocietyArtificial IntelligenceComputation and Language
Summary
People are starting to rely on AI language models like ChatGPT for answers to personal, subjective questions as if the AI knows everything. The authors studied how and why people use these models as all-knowing oracles, finding that younger people do this more often and that people often don’t realize they are doing it. They created tools to analyze AI usage while protecting privacy and found users sometimes feel unhappy about depending on AI for personal decisions. The authors also identified factors that make people rely on AI like their beliefs about AI and how the AI responds.
What this means in practice
- •For ai product designers: Design user interfaces that alert when users rely too much on AI for personal judgments and support their self-reflection.
- •For privacy tool developers: Build privacy-preserving analytics tools to study how individuals interact with AI over time without compromising user data.
Authors
Myra Cheng, Lujain Ibrahim, Grace Liu, Michelle S. Lam, Vishakh Padmakumar, Nick Madibekov, Diyi Yang, Dan Jurafsky
Abstract
We characterize how people are turning to LLMs as oracles: all-knowing authorities on subjective personal questions. Motivated by risks to users' autonomy and well-being, we develop a typology and LLM-based methods to measure this form of AI reliance at scale and understand how people are offloading judgment and decision-making to AI. Applying our typology to public usage data (68K prompts from WildChat and ThoughtTrace), we find that LLM-as-oracle use has increased over time (2023-2026) and is more prevalent among younger users. We further build a privacy-preserving data donation tool to analyze individuals' longitudinal usage data (140K prompts from 52 participants), identifying similar trends. People are often unaware of their own LLM-as-oracle use, and express dissatisfaction with this behavior after seeing our tool's analysis. Finally, we identify two drivers of LLM-as-oracle use: people's perceptions of AI and the behavior of AI models themselves, which motivate possible interventions to support users' self-deliberation.