Text-Guided Diffusion-Based Adversarial Attacks on Chest X-Ray Images
Computer Vision and Pattern Recognition
Summary
The authors studied how AI models that read chest X-rays can be tricked in ways that look real but make the AI give wrong answers. Instead of just changing pixels randomly, they used a method that changes images in a more natural way guided by text, keeping the AI that makes images and the X-ray classifier fixed. They found their method fooled AI models much more while the images still looked accurate to doctors, who mostly saw no change. This shows AI can be vulnerable to subtle but realistic-looking changes, suggesting that testing AI safety needs to include these smarter attacks.
Authors
Basudha Pal, Arjun Narayanan, Neha Ajith, Vikas R Bhat, Muhammad Umair
Abstract
As artificial intelligence is increasingly integrated into chest X-ray (CXR) interpretation, triage, and clinical decision support, understanding its vulnerability to adversarial manipulation is critical for safe deployment. Existing robustness evaluations, however, predominantly rely on pixel-space attacks that introduce numerically constrained perturbations but may not represent plausible radiographic variation. This limitation is particularly important in multi-disease CXR classification, where models simultaneously evaluate multiple overlapping pathologies and adversarial failures may alter several diagnostic predictions. We propose a text-guided diffusion-based adversarial framework that optimizes learnable text conditioning while keeping the diffusion generator and target classifier frozen, enabling adversarial generation through a learned image prior rather than direct pixel manipulation. We evaluate the framework across multiple classifier architectures in both binary atelectasis and multi-disease CXR classification and compare it with FGSM, PGD, and Carlini-Wagner attacks. Our approach consistently produced the greatest degradation in classifier performance, reducing AUROC to 0.3885-0.5646 in binary classification and 0.4441-0.4878 in the multi-disease setting, while achieving superior image fidelity (SSIM 0.9080, LPIPS 0.1670, FID 51.23). Importantly, clinician interpretation remained unchanged for 95.9% of binary and 73.8% of multi-disease adversarial images despite substantial changes in model predictions. These findings reveal a clinically important discrepancy between human and machine interpretation and demonstrate the need to extend medical AI robustness evaluation beyond conventional pixel-space attacks toward generative threat models that can expose failures under visually and clinically plausible image variations.