Summary
Predicting what will happen when you change something in the world is different from just guessing based on past observations. The authors show that models can be good at prediction but still make mistakes when trying to understand cause and effect if they only learn from data without knowing the right causal story. They explain why it's impossible for such models to always get cause-and-effect relationships right unless they consider better explanations about how the world works. The authors suggest that to truly understand interventions, models need to actively test and update their ideas about how things cause one another.
What this means in practice
- •For ai system developers: Identify the limits of predicting intervention effects from data-only training to improve AI system design for tasks needing causal understanding.
- •For autonomous robotics teams: Design robots that can test and revise their causal models actively to better handle interventions and changes in the real world.
A theory result. No direct application yet.
Abstract
Retaining predictive gains can require persistent errors in a model's response to interventions. For deterministic predictors with finite second moments under squared-error loss, any observational advantage over the best predictor using only an outcome's direct causes requires dependence on other inputs, regardless of the learning rule. When admissible interventions vary inputs while preserving the worldly outcome, this dependence yields a positive lower bound on root-mean-square intervention-response error. We identify conditions under which population gradient updates introduce this dependence from a parent-only baseline or damage a previously correct mechanism through shared parameters while improving other predictions. These dependencies can also make model behavior vary with how the same intervention is realized. We characterize interventional alignment through a common output mapping across internal states and physical processes realizing the same intervention target and relevant context. Correction is limited by the causal hypotheses currently admitted within inquiry. The relevant hypothesis class comprises these causal alternatives, rather than the function class the computational system could in principle represent. For predictions restricted to these alternatives, interventional mean error cannot fall below the best approximation they permit, whether evidence is observational or interventional. Reducing error below a positive limit requires admitting better hypotheses, which may entail revising the causal variables, mechanism structure, or intervention interpretation. Interventional criticism constrains replacements without determining them. We conjecture that reliably acquiring and maintaining interventional competence when causal commitments fail requires autonomous participation in proposing alternatives, testing their consequences, and revising them in response to error.