Xglass prioritizes key ct slices using x-ray gaze and language data
XGLASS: X-Ray, Gaze and Language Assisted Slice Sequencing for CT Volumes
Computer Vision and Pattern Recognition
Summary
Reading all the slices of a CT scan takes a lot of time because there are many images to check. The authors present XGLASS, a method that uses how doctors look at 2D X-rays along with language describing diseases to figure out which CT slices are most important. This lets doctors see only a small number of key slices that are most useful for diagnosis. The method reduces the effort and computing power needed while still catching important medical findings effectively.
What this means in practice
- •For medical imaging developers: Reduce the number of CT slices needing detailed analysis while improving diagnostic slice selection accuracy using cross-modal gaze and language-informed prioritization.
- •For radiology workflow managers: Speed up CT scan interpretation by preselecting key slices that likely contain important findings, cutting computing time and resource use substantially.
Authors
Michael Glöckler, Timo Ropinski
Abstract
Vision-language models (VLMs) achieve remarkable performance on 2D medical images, such as X-rays or pathology images, while 3D VLMs that directly process CT/MRI volumes lag behind due to limited 3D training data and the predominance of 2D foundation architectures. To leverage powerful 2D VLMs for analyzing medical volumes, which contain hundreds of slices, it is therefore necessary to focus only on those few slices, which are diagnostically relevant. To enable such a prioritization, we propose XGLASS, a method that transfers medical expertise from 2D X-rays to 3D CT for task-aware slice prioritization. XGLASS uses disease-conditioned importance maps derived from language-aligned radiologist gaze on X-rays and spatially aligns them with coronal CT projections, bringing gaze priors and CT content into a shared 2D representation. The fused gaze-prior/CT projection is then used to predict a disease-specific ordering of axial slices, prioritizing diagnostically relevant slices. XGLASS significantly improves budgeted retrieval over 2D and 3D-native baselines, achieving a top-10 hit rate up to 75.9% for the best-performing disease. In practice, this means that after inspecting only the top 10 slices (4.17% of a 240-slice scan), XGLASS identifies a diagnostically relevant slice in more than 75% of cases. For downstream slice-wise finding classification, an XGLASS-selected 30-slice candidate set reduces model calls by 87.5% while improving F1 by 11.3 percentage points over all-slice inference, 4.2 points over the strongest 2D selector, and remains competitive with 3D-native selectors while requiring up to 130.8 times less time and 10.9 times less GPU memory per finding, supporting its use as a lightweight preselector for volumetric CT analysis. The code and pretrained checkpoint are available on https://github.com/michaelgloeckler/XGLASS.