Whole slide image answering improves by preserving and focusing on morphology
Look Beyond the Question: Preserve-Focus-Fuse for Whole-Slide Image Question Answering
Computer Vision and Pattern Recognition
Summary
Answering questions about very large medical images is tricky because the questions don't always mention all the details needed to find the right answers. The authors created a new method called BEYONDPATH that looks at the entire slide to remember important tissue features, then zooms in to find specific clues for each question, and finally combines this information carefully. This approach helped them perform better than other methods on tests, showing that keeping broad context around the slide is important even when looking for specific answers.
What this means in practice
- •For medical imaging software developers: Build tools that answer pathology questions by preserving detailed tissue patterns while searching regions relevant to specific queries.
- •For digital pathology service providers: Improve automated reporting systems by integrating broad slide morphology with question-focused retrieval for better diagnostic support.$Commercial implications: Enables selling more accurate AI-assisted pathology reporting services that combine comprehensive context with targeted slide analysis.
Authors
Xiangyu Li, Jincheng Li, Tianle Du, Yanheng Li, Haoxuan Che, Yuxin Sun, Long Bai, Liangliang Liu, Ran Su
Abstract
A question about a whole-slide image rarely names every morphological finding needed to answer it. Question-guided patch selection can therefore overlook supporting morphology, while slide-level compression can dilute focal evidence. Inspired by how pathologists combine broad slide review, question-directed examination, and integration of their findings, we introduce BEYONDPATH, a Preserve-Focus-Fuse framework for whole-slide image question answering. Preserve learns a question-agnostic Morphology Bank through answer supervision across multiple questions about each training slide. Focus retrieves questionspecific evidence from all foreground patches, including regions outside the Bank. Fuse integrates the preserved morphology at the retrieved evidence positions through Morphology-on-Visual-Evidence Fusion, producing a fixed 96-token visual prefix. On WSI-Bench, BEYONDPATH achieves an Avg-9 of 0.769 and leads all eight reported report-generation metrics using the same checkpoint. It also achieves 0.639 zero-shot overall accuracy on SlideBench-BCNB. Controlled ablations show complementary benefits from morphology preservation and question-directed retrieval, and support cross-question supervision and fusion at the evidence positions. These results support looking beyond the question's explicit cues: preserving broader slide morphology while retaining the ability to seek evidence for each question.