Selectively sharing information improves human decisions under budget constraints
Should I stay or should I show? Learning to selectively disclose information
Machine Learning
Summary
Sometimes people get extra information to help make big decisions, but getting and sharing this info costs time and effort, and can sometimes confuse rather than help. The authors studied how to decide when it's best to share helpful information and when it's better not to. They found a smart rule based on weighing the value of the information against costs and risks. Their experiments showed sharing only useful info at the right times helps people make better choices compared to always sharing or never sharing. They also tested this with real human teams and found their approach often improves teamwork between people and AI.
What this means in practice
- •For medical decision teams: Provide doctors with diagnostic info only when it statistically reduces their chance of error under time or cost limits.
- •For customer support centers: Help support agents get background info on cases selectively to improve resolution quality without overwhelming them.
Authors
Carlotta Giacchetta, Alessando Bogani, Cesare Barbera, Giovanni De Toni, Michele Caprio, Andrea Pugnana, Andrea Passerini
Abstract
In many high-stakes settings, human decision-makers can acquire support information before making a decision. However, acquiring information is costly, and disclosure may fail to improve human decisions or may even impair them. We tackle this problem by studying selective disclosure, i.e., the problem of learning when to reveal support information to a human decision-maker under a budget constraint. We first show that the optimal policy is a threshold rule on the Value of Information (VoI), i.e., the expected reduction in human decision risk induced by disclosure. Since VoI is unknown in practice, we estimate the regime-specific human risks and bound the possible degradation of the resulting plug-in policy relative to lack of disclosure, as well as its regret relative to the optimal policy. Experiments on benchmark datasets show that selective disclosure outperforms both no disclosure and full disclosure, regardless of whether the support information is beneficial or harmful. Two user studies show that human-AI team performance can improve when disclosure is led by our learned policy and not human-selected, although this advantage varies across tasks. A counterfactual benchmark, which replaces participants' predictions with a machine-learning prediction when disclosure occurs, suggests that these differences might depend on lower adherence to advice when the information is automatically provided rather than self-requested.