Feedback shapes how humans work with AI advice effectively
Think Once, Reflect Twice: How Feedback Shapes Human-AI Collaboration
Human-Computer Interaction
Summary
Deciding when and how much to trust AI help is tricky. This paper shows that giving people balanced feedback about when they override AI advice helps them think about their choices afterward without hurting their thinking during decisions. The authors found that one-way feedback can reduce careful thinking while deciding and shift trust in AI in ways that don't improve overall decision quality. Balanced feedback helps people reflect both during and after decisions, leading to better teamwork with AI.
What this means in practice
- •For healthcare teams: Improve medical decisions by designing feedback systems that encourage doctors to thoughtfully assess AI advice during and after patient diagnosis.
- •For financial advisors: Create feedback tools that help advisors evaluate when to follow or override AI recommendations for better investment choices.
Authors
Joshua Holstein, Philipp Spitzer, Sebastian Schäfer, Maximilian Förster, Gerhard Satzger
Abstract
As artificial intelligence (AI) increasingly supports human decision-making in high-stakes domains, designing effective human-AI collaboration becomes critical. Successful collaboration requires humans to reflect on decision-making processes and adjust their strategies accordingly. Current research typically investigates interventions, such as explainable AI, that target reflection during the decision-making process (reflection-in-action). Less attention has been given to retrospective examination of decisions (reflection-on-action). This work explores reflection-on-action by studying the impact of feedback on reliance behavior and human-AI team performance. Specifically, we investigate feedback interventions targeting overrides of AI advice through a behavioral study (N=244). We find that such feedback influences reflection-on-action, while unidirectional feedback risks reducing subsequent reflection-in-action. Furthermore, we identify opposing shifts in reliance behavior that leave overall human-AI team performance unchanged but show context-dependent effects. Balanced feedback did not show these risks, preserving both concurrent and retrospective reflection, offering insights for designing for appropriate reliance on AI.