Deep learning classifies kidney tumors from CT scans without detailed labels
Weakly-supervised Kidney Tumor Classification from CT Scans with Multi-Instance Learning and Anatomical Filtering
Computer Vision and Pattern Recognition
Summary
Detailed labeling of CT scans by doctors is hard and time-consuming, so the authors studied ways to train computer models using only overall scan labels rather than exact tumor outlines. They split CT scans into smaller 2D images to make training easier and used a method to automatically focus on important body parts without needing exact segmentations. The authors tested two methods to identify kidney tumors from CT scans and achieved good accuracy using only scan-level information. Their approach also helped the models work well on different datasets, showing potential for analyzing other diseases and organs.
Computed tomography (CT)Kidney tumor classificationMulti-instance learning (MIL)Weak supervisionAnatomical filteringBody part regressionAttention-based MILFocusMILImageNet pretrainingOut-of-distribution generalization
Authors
Joonas Ariva, Dmytro Fishman
Abstract
Deep learning models for CT scan analysis are often limited by the scarcity of precise pixel-level annotations, which require significant radiologist effort to produce. Training on scan-level labels alone reduces annotation requirements but introduces challenges: low supervision ratios and large input volumes make models prone to overfitting and shortcut learning. In this work, we investigate two complementary methods to address these challenges: multi-instance learning (MIL) and anatomical filtering. MIL divides CT volumes into 2D slice instances, enabling efficient 2D architectures with ImageNet pretraining rather than computationally demanding 3D models. Anatomical filtering uses Compass, our self-supervised body part regression model, to crop scans to pathology-relevant subregions without requiring segmentation masks. We evaluate two MIL frameworks - Attention-based MIL (ABMIL) and FocusMIL - on kidney tumor classification across one internal dataset (TUH) and two external datasets (KiTS23 and TCGA-KiRC). Our best models achieve F1 = 0.83 on the internal test set using only scan-level labels. We further show that anatomical filtering with the Compass model is critical for the out-of-distribution generalization of embedding-based ABMIL, while instance-based FocusMIL demonstrates greater inherent robustness to distribution shift. While evaluated on kidney tumors, we consider this a proof-of-concept for a broader weakly supervised CT classification pipeline applicable to other organs and pathologies.