Vision transformers read text annotations instead of x-ray images
Spurious Attention Heads in Vision Transformers: Mechanistic Identification of Annotation-Driven Bias in Medical Chest X-Ray Analysis
Computer Vision and Pattern RecognitionComputation and Language
Summary
Vision transformer models designed to analyze chest X-rays sometimes focus on the text annotations in the image corners instead of the lung areas with disease. The authors found a specific part of the model that pays attention mostly to these texts, leading to misleading results. They showed that this behavior happens consistently across different models and datasets, and fine-tuning the models does not fix the issue. Removing this attention to annotations increases focus on relevant lung regions, but this behavior is not obvious from normal testing.
What this means in practice
- •For medical ai developers: Identify and remove attention mechanisms that exploit text annotations, improving model reliability on medical images.
- •For machine learning engineers: Audit vision transformer models mechanistically to detect hidden biases not visible in output-level testing.
Authors
Joshua Muhumuza, Cornelius Kalenzi
Abstract
Vision Transformers deployed in medical imaging exploit spurious correlations, yet the internal mechanisms remain unidentified and impossible to fix. We apply mechanistic interpretability to locate one such mechanism in ViT-Base 16: a single attention head (L8H10) that functions as a dedicated annotation-reader on chest X-rays, attending to corner text rather than lung pathology in 62% of 200 COVID-19 images. Causal validation via inpainting confirms annotation content rather than luminance or edge density drives the behaviour, increasing lung attention 1.52 times upon removal in affected images ($p{<}0.0001$). The finding replicates across NIH ChestX-ray14 (79%), DeiT-Base (88%), and ViT-Large 16 (80%), establishing spurious annotation-attending heads as a consistent property of ViT-family architectures across two datasets and three architectures. Full fine-tuning does not suppress the behaviour and may entrench it: 98.4% of spurious images remain spurious and the spurious rate rises marginally, as the shortcut is reinforced rather than removed. Critically, single-head ablation produces $p{=}0.340$ and multi-head ablation of all three identified carriers produces $p{=}0.532$, meaning a causally verified annotation-reading circuit is completely invisible to output-level evaluation. Mechanistic auditing finds what behavioural testing cannot.