Language model proxies fall short of human reliance on ai answers
Can LLM User-Proxies Stand In for Humans in Reliance Measurement? Replicating Rel-A.I. with Simulated Users
Artificial Intelligence
Summary
This paper looks at whether language models can act like humans in deciding how much they trust AI answers. The authors check earlier studies and use data from a past human experiment called Rel-A.I., but some key details from humans were missing. They find that language-model user proxies don’t perfectly match human reliance patterns, especially across different topics. The study shows that while proxies can imitate some trends, they still have big differences from humans in how much they rely on AI across various domains.
What this means in practice
- •For ai system evaluators: Assess AI user reliance more accurately by knowing language-model proxies do not fully replicate human decisions across topics.
- •For ai product designers: Improve AI interface designs by recognizing the limits of simulated users in predicting real human trust and reliance.
Authors
Shuai Wang, Rui Hu
Abstract
Can language-model user proxies reproduce human reliance on an AI answer across contexts? Published simulated-user studies do not establish validity for this partial-answer decision, and Rel-A.I.'s released human record omits trial-level choices. We audit its 90 printed questions and 15 domain-by-marker reliance rates using sampled decisions and counterbalanced A/B probabilities from open models through 72B parameters. A matched crossover varies the printed agent specialty within each question, separating question content, label averages, and their interaction. Mistral Small's strong-marker math-minus-law label marginal is -0.4 percentage points, yet matching the agent label to the question domain raises reliance by 13.3 points on average across the five domains. The best A/B proxy has 22.5-point absolute-rate error and 8.3-point domain-effect error against the human table. A two-sample independent-binomial human replication has 3.5-point expected domain-effect error under the stated working model. The interaction is model dependent, and inference layout changes identical-prompt probabilities. This single-study audit does not establish proxy validity for human effect sizes or general directional screening.