Multi scale fusion improves prostate cancer grading accuracy
Multi-scale Fusion of Fast Vision Mamba and Foundation Model Features for Patch-Level Gleason Grading in Computational Pathology
Computer Vision and Pattern Recognition
Summary
Grading prostate cancer from microscope images is tough because different cancer types can look very similar and important details appear at different zoom levels. The authors created a new method that combines information from two types of computer models working at different scales to better recognize these patterns. Their approach improved accuracy on a large prostate cancer image dataset, especially for rare and subtle cancer types. This shows that looking at cancer pictures in multiple ways helps computers grade them more reliably.
What this means in practice
- •For medical imaging teams: Automatically assign prostate cancer grades from pathology slides with improved accuracy by combining multi scale features and models.
- •For digital pathology platform developers: Integrate multi scale model fusion techniques to enhance diagnostic support tools for prostate cancer grading in clinical workflows.
Authors
Zihe Zhang, Weimiao Yu, Piumi Don Simonge
Abstract
Automated Gleason grading of prostate whole slide images (WSIs) enables faster and more consistent cancer risk stratification. However, it remains a challenging task due to the subtle morphological differences between grades, the severe class imbalance arising from rare patterns, and the diagnostic features that span multiple spatial scales. Pathology foundation models (FMs) extract strong patch representations but often operate at a single-scale, which limits spatial context and disproportionately affects classes that depend on broader context. The challenge of fusing multiple scales and complementary models, with minimal data and computational resources, remains unresolved. We propose a multi-scale classifier built on state-space models. Our method combines two complementary feature sources: a frozen foundation model, and a lightweight FastVim, pre-trained on ImageNet and distilled from the foundation model, that reads the raw image crops. At each scale, a Mamba model encodes the features from both sources with linear-time spatial context, and a jointly trained head fuses the encoded representations into one prediction. On the PANDA dataset, our method reaches macro F1 of 0.894 and Gleason 5 F1 of 0.802 under 5-fold cross-validation, a 12% and 21% relative improvement over the best single-scale model. A controlled study over 5 foundation models, 5 base classifiers and 3 scales confirms that the gain holds across encoders, with larger improvements observed at smaller scales.