Improving chest X-ray AI by fixing visual and semantic mismatches
PLRS-IC: A Dual-Calibration Framework for Chest X-Ray Vision-Language Alignment
Computer Vision and Pattern Recognition
Summary
Chest X-rays come from different angles and show different views of the same problem, which makes it hard for AI to understand them the same way. Also, similar medical conditions in different patients can confuse the AI when it tries to tell images apart. The authors created a two-step method that adjusts how the AI matches parts of the image to words, considering the angle of the X-ray, and also makes it less harsh when comparing images that share similar medical meanings. Their method helps AI do better at recognizing, locating, and outlining problems in chest X-rays without needing extra labeled examples.
What this means in practice
- •For medical imaging engineers: Enhance chest X-ray analysis tools to recognize and localize diseases without extra labeled training data by using projection and semantic calibration.
- •For healthcare data scientists: Improve zero-shot interpretation of diverse chest X-ray views by reducing errors from visual and semantic mismatches in training data.
Authors
Qixing Zhao, Jinpeng Li
Abstract
Fine-grained vision-language alignment in chest radiography enables zero-shot classification, grounding, and segmentation without task-specific annotations. However, this alignment is fundamentally hindered by two intertwined sources of ambiguity: projection-induced visual mismatch and patient-agnostic semantic overlap. First, at the local feature level, frontal and lateral radiographs exhibit distinct appearances for the same clinical finding, rendering a shared patch-text similarity geometry inherently suboptimal. Compounding this visual ambiguity is a semantic mismatch during global contrastive optimization, where instance-level objectives penalize cross-patient pairs as strict negatives even when they share identical positive clinical concepts. To address this dual ambiguity, we propose PLRS-IC, a unified dual-calibration framework for chest X-ray representation learning. At the local alignment stage, Projection-Conditioned Low-Rank Residual Similarity (PLRS) dynamically adapts patch-text matching to projection-specific manifolds using a bounded, parameter-efficient low-rank residual. At the global optimization stage, Information-Content-Calibrated Soft False-Negative Suppression (IC-SFNS) leverages a corpus-derived information-theoretic prior to soften the penalty of semantically overlapping negatives without altering original contrastive assignments. Extensive experiments across nine public zero-shot benchmark settings demonstrate that our framework yields consistent improvements in classification, grounding, and segmentation, validating the necessity of dual-calibration in medical vision-language pre-training.