Pathology foundation models enable case retrieval for rare tumors
Pathology foundation models for training-free case retrieval in salivary gland tumors
Computer Vision and Pattern Recognition
Summary
Recognizing rare types of salivary gland tumors is hard, even for experienced pathologists. The authors tested if powerful image understanding models for pathology can help by retrieving similar past cases without needing extra training. By selecting an area on a tumor slide, pathologists can find similar cases from a digital archive quickly. This makes it easier to compare and diagnose unusual tumors based on similarities to known examples, expanding the knowledge pathologists rely on.
What this means in practice
- •For pathology departments: Make large archives of digitized tumor slides searchable by image similarity to aid diagnosis of rare tumor types without extra training.
- •For medical image software developers: Implement image-based query tools for histopathology scans that allow users to retrieve similar cases using pretrained foundation models without task-specific tuning.
Authors
Maximilian J. Roesel, Niklas Kiermeyer, Philipp Keyl, Frederick Klauschen
Abstract
In pathology, pattern recognition relies on a frame of reference built through extensive training and experience, which may still fall short when encountering rare cancer subtypes. Archives of previously diagnosed tumors often contain relevant precedents, yet identifying the most relevant cases remains challenging. Here we explore whether content-based retrieval with current pathology foundation models facilitates rare cancer diagnostics without any task-specific training. Pathologists can query a digitized archive using a region of interest (ROI) from a whole slide image (WSI) and retrieve morphologically similar cases, along with their ICD-O codes and corresponding WSIs. The ROI can be selected manually or automatically using a gated attention-based multiple instance learning model. We evaluate our approach by leave-one-case-out on an imbalanced dataset of 137 salivary gland tumor cases spanning 12 ICD-O morphology categories, comparing four pathology foundation models (Atlas2, Atlas2-B, Atlas2-S, and UNI2h). Top-1/3/5 retrieval was 62.5%/76.7%/81.3% for Atlas2, 50.2/66.6/72.9% for UNI2h, 52.3%/71.5%/77.6% for Atlas2-B and 54.4/65.2/73.7% for Atlas2-S. Macro-averaged F1 ranged from 69.0% for the smallest model Atlas2-S to 76.6% for the largest model Atlas2 across the five classes with at least ten cases. Our modified Chamfer similarity separated correct from incorrect matches among the Top-5 more reliably for Atlas2 (AUC 0.754) than for Atlas2-B (0.679, p = 0.020) and Atlas2-S (0.653, p = 0.003), but not UNI2h (0.697, p = 0.196). Requiring no additional training or manual annotation, this approach makes departmental archives searchable with pathology foundation models, providing a low-barrier route to case-based differential diagnosis and extending the individual frame of reference to the collective experience of the department.