AI advice can help or harm decision quality depending on context
Does AI Help or Harm? Endogenous Information Acquisition with AI Advice
Computer Science and Game Theory
Summary
When people get advice from AI before making expensive decisions, it can change how well they decide. The authors studied when AI advice improves or worsens decision outcomes. They found that in simple situations with two possible states, AI advice never hurts. But in more complex cases, AI advice can sometimes stop people from gathering important information, lowering decision quality. This shows AI assistance is not always helpful and depends on the details of the decision environment.
What this means in practice
- •For decision support designers: Design AI advice systems that avoid reducing valuable information gathering in complex decisions with multiple possible outcomes.
- •For financial risk analysts: Assess when AI signals may inadvertently cause harm by prematurely stopping costly but valuable data collection in risk evaluation.
Authors
Kexin Chen, Tao Lin, Jianwei Huang
Abstract
AI advice increasingly enters human decision workflows before costly information acquisition and final action. This paper studies how such advice affects decision quality, measured by the gross utility of the final action. Advice shifts beliefs before the decision maker chooses an acquisition experiment with uniformly posterior-separable (UPS) costs. The analysis characterizes gross effects through the continuation gross value generated by net-optimal acquisition. Convexity of this value function is necessary and sufficient for every AI signal to weakly improve gross utility at every prior, while nonconvexity allows a binary signal to cause gross harm. The criterion gives tight state-space results. Every AI signal is weakly gross-improving in binary-state problems under arbitrary actions, payoffs, and UPS costs. A minimal three-state problem can generate gross harm through concentrated shutdown, where valuable acquisition stops. For classification problems with Shannon acquisition costs, the criterion becomes an active-set slope test. It guarantees weak gross improvement for homogeneous classification and all three-state weighted classification problems, while a minimal four-state failure arises from attention dilution, where acquisition shifts away from high-stake distinctions. Along a Blackwell-increasing mixture path, gross utility can first decrease and then increase. With endogenous interpretation, cheaper interpretation can reduce gross utility.