Robust method finds changes to improve decisions despite uncertainty
Robust Counterfactuals for Markov Decision Processes under Statistical and Causal Uncertainties
Artificial Intelligence
Summary
Understanding what could have happened if different choices were made is important for improving AI decisions in tasks that involve many steps. This is tricky because data is incomplete and there can be multiple explanations for why things happen. The authors introduce a way to handle both data uncertainty and different possible cause explanations at once. Their method calculates reliable bounds on what could happen and identifies the fewest changes needed to improve outcomes. Tests show it works well in different scenarios.
What this means in practice
- •For ai safety teams: Find minimal action changes to prevent failures in AI systems under uncertainty.
- •For operations researchers: Improve decision-making by robustly identifying alternative strategies when data and cause models are uncertain.
Authors
Jessica Lally, Milad Kazemi, Nicola Paoletti
Abstract
Counterfactual reasoning (asking what would have happened, had a decision been different) is a powerful tool for identifying causes of failure and explaining the behaviour of AI systems in sequential decision-making tasks. In Markov Decision Processes (MDPs), two sources of uncertainty complicate counterfactual inference: statistical uncertainty in the MDP transitions when the MDP is estimated from trajectory data, and uncertainty about the unknown causal model, as there are multiple structural causal models (SCMs) compatible with each MDP. Existing work addresses these two challenges in isolation. We address both simultaneously by introducing a framework for partial counterfactual inference in MDPs that is robust to both statistical and causal uncertainty. Our approach provides: 1) closed-form analytical bounds for the counterfactual probabilities that are valid across all SCMs entailed by a given interval MDP (IMDP); 2) a method to compute minimal robust interventions to achieve a desired counterfactual outcome via pessimistic value iteration on the induced counterfactual IMDP. Since the latter is constructed using our analytical bounds, our approach remains efficient and scalable to realistic MDP environments. Experiments across multiple case studies demonstrate our method's robustness and practical utility for identifying minimal sets of actions that, if changed, would have led to improved outcomes.