Attention guidance improves reliability in multiple instance learning tasks
CAR-MIL: Counterfactual Attention Regularization for Multiple Instance Learning
Computer Vision and Pattern Recognition
Summary
When computers are trained to recognize details in images without exact labels, they can get confused about what parts of an image really matter. The authors designed a new way to teach the computer to pay attention to important areas more accurately by showing it what would happen if it shifted its focus slightly. This approach leads to better detection and explanation of relevant image parts, especially in complex medical images. Their method performs well on test data and makes the computer's focus more trustworthy.
What this means in practice
- •For medical image analysts: Provide more reliable attention maps to help identify important regions in digital pathology images with limited annotations.
- •For machine learning engineers: Improve weakly supervised models by integrating counterfactual reasoning to refine attention allocation and model explanations.
Authors
Imane Chraki, Pierre Marza, Stergios Christodoulidis, Maria Vakalopoulou
Abstract
Multiple Instance Learning (MIL) is widely used for weakly supervised learning, particularly in digital pathology, where fine-grained annotations are costly. Most MIL methods aggregate instance features via attention mechanisms. However, attention weights do not always faithfully reflect instance importance and may focus on spuriously correlated regions. In this work, we propose CAR-MIL, a framework that explicitly guides attention learning through a counterfactual attention regularization objective inspired by counterfactual explanations. Built on a standard attention-based MIL architecture, our approach introduces a lightweight counterfactual attention branch trained to produce an alternative prediction while remaining close to the factual attention distribution. This encourages prediction changes to arise from minimal, structured redistributions of attention, leading to more informative evidence allocation. The resulting factual and counterfactual attention maps capture complementary evidence: the former highlights regions supporting the prediction, while the latter reveals regions whose reweighting would challenge it. We evaluate our method on synthetic MIL benchmarks with instance-level ground truth enabling controlled analysis of attention behavior and on five digital pathology datasets across four tasks. CAR-MIL maintains competitive classification performance, with the largest gains observed on more challenging tasks, while improving attention reliability, demonstrating the benefits of integrating counterfactual explainability reasoning into attention learning. Code is available at: https://github.com/ImaneCR/CAR-MIL/.