PathPilot improves slide navigation by adapting magnification and focus
PathPilot: An Adaptive Question-Conditioned Multi-Magnification Navigation Agent for Whole-Slide Image Analysis
Computer Vision and Pattern Recognition
Summary
Looking at medical slides to find disease signs is tricky because doctors zoom in and out and move around to gather clues. Existing AI tools usually only look at one zoom level or follow fixed zoom patterns, missing how real doctors explore slides. The authors created PathPilot, an AI that learns where to look and how much to zoom based on how pathologists view slides and the questions it needs to answer. PathPilot moves step-by-step, changing zoom and area intelligently, gathering enough evidence for diagnosis like a doctor would. Tests show PathPilot works better than previous methods on different patient groups.
What this means in practice
- •For clinical ai developers: Develop AI systems that navigate medical slides dynamically across magnifications to improve diagnostic efficiency and accuracy.
- •For medical imaging software teams: Integrate adaptive multi-scale navigation tools into digital pathology platforms to mimic pathologists' smart slide examination routines.
Authors
Zhengyang Xu, Han Li, Jingsong Liu, Ayako Ura, Shihui Zu, Shaohua Kevin Zhou, Nassir Navab, Peter Schüffler
Abstract
The diagnostic capability of WSI agents relies on effective slide navigation to identify diagnostically relevant regions and efficiently integrate evidence across the slide. While in clinical practice, pathologists dynamically examine slides across multiple scales, integrating global and cellular evidence in a sequential manner, existing AI navigation approaches mostly operate at a single fixed magnification or rely on predefined magnification traversal. This mismatch prevents existing methods from modeling cross-magnification interactions and adaptive magnification selection inherent to real diagnostic workflows. We propose PathPilot, a clinically consistent multi-magnification WSI navigation agent that explicitly models cross-magnification localization and evidence-adaptive exploration. Specifically, we introduce a CrossMagLoc that learns magnification-specific diagnostic localization from pathologist viewing behavior and integrates information from adjacent magnifications to identify the diagnostically most relevant region within the current ROI, thereby providing a diagnostically informed starting point for subsequent question-conditioned exploration. We further introduce an EviRegionSearch that starts question-conditioned exploration from this region, progressively accumulating question-relevant evidence and adaptively determining the next navigation action according to the remaining evidence requirement. Their iterative interaction enables PathPilot to sequentially explore across ROIs and magnifications until sufficient evidence is acquired. Extensive QA evaluations across five cohorts demonstrate consistent superiority over competing SOTA methods on inter/external cohorts, respectively, validating question-conditioned evidence acquisition across regions and magnifications that mirrors the sequential, knowledge-driven navigation of pathologists.