Attention focused better in weakly labeled pathology images using counterfactual learning
CAR-MIL: Counterfactual Attention Regularization for Multiple Instance Learning
Computer Vision and Pattern Recognition
Summary
It can be hard to teach computers to recognize important details in medical images because the exact parts to look at are often not clearly labeled. The authors created a new method called CAR-MIL that helps the computer’s attention system learn better by comparing what changes when different clues are focused on. This makes the computer highlight areas that truly support its decisions and also shows what parts would challenge its guesses if weighted differently. Their method was tested on fake and real medical image data and improved the reliability of where the computer looks without losing accuracy.
Multiple Instance Learningattention mechanismcounterfactual explanationsdigital pathologyweak supervisioninstance featuresmachine learningexplainability
Authors
Imane Chraki, Pierre Marza, Stergios Christodoulidis, Maria Vakalopoulou
Abstract
Multiple Instance Learning (MIL) is widely used for weakly supervised learning, particularly in digital pathology, where fine-grained annotations are costly. Most MIL methods aggregate instance features via attention mechanisms. However, attention weights do not always faithfully reflect instance importance and may focus on spuriously correlated regions. In this work, we propose CAR-MIL, a framework that explicitly guides attention learning through a counterfactual attention regularization objective inspired by counterfactual explanations. Built on a standard attention-based MIL architecture, our approach introduces a lightweight counterfactual attention branch trained to produce an alternative prediction while remaining close to the factual attention distribution. This encourages prediction changes to arise from minimal, structured redistributions of attention, leading to more informative evidence allocation. The resulting factual and counterfactual attention maps capture complementary evidence: the former highlights regions supporting the prediction, while the latter reveals regions whose reweighting would challenge it. We evaluate our method on synthetic MIL benchmarks with instance-level ground truth enabling controlled analysis of attention behavior and on five digital pathology datasets across four tasks. CAR-MIL maintains competitive classification performance, with the largest gains observed on more challenging tasks, while improving attention reliability, demonstrating the benefits of integrating counterfactual explainability reasoning into attention learning. Code is available at: https://github.com/ImaneCR/CAR-MIL/.