Cytospm improves detection of diverse cell types in cytopathology images
CytoSPM: Open-Vocabulary Cytopathology Detection with Structured Prompt Bank
Computer Vision and Pattern Recognition
Summary
Detecting different types of cells in medical images of body fluids is hard because there are many tiny and similar-looking cell types which keep changing depending on the organ. The authors created a new test called PentaCyto that covers five different body fluids and groups cell types into known and new ones. They designed a tool called CytoSPM that first finds general visual features in images and then matches them with detailed text descriptions of cell shapes and features. CytoSPM works better than previous methods at spotting new cell types accurately and quickly on this test.
What this means in practice
- •For clinical laboratory teams: Improve automated detection of a wide range of cell types in cytopathology images across multiple body fluids for faster and more accurate diagnoses.
- •For biomedical software developers: Build cytopathology analysis tools that can recognize evolving and rare cell types by integrating structured text prompts describing cell morphology.
- •For medical imaging companies: Develop commercial diagnostic imaging products that identify diverse and novel cytopathology cell categories utilizing the CytoSPM open-vocabulary detection approach.$Commercial implications: It enables detection of novel and fine-grained cell types in cytopathology images, enhancing diagnostic capabilities of imaging software sold to hospitals.
Authors
Wenjie Li, Zishan Xu, Jinyang Huang, Zhengxin Nie, Shichao Kan, Yixiong Liang
Abstract
Cytopathology detection requires open-vocabulary recognition because cellular categories are fine-grained, long-tailed, and continuously evolving across different organ systems. However, existing cytology detectors are mostly single-domain and closed-set, and there is still no unified benchmark for evaluating open-vocabulary cytopathology detection. We present PentaCyto, a multi-domain benchmark covering cervical, urinary, respiratory, serous fluid, and thyroid cytology, with 24 base categories and 9 held-out novel categories. Each category is associated with structured cytomorphology prompts that describe diagnostic morphological attributes and provide clinically grounded textual knowledge. We further propose CytoSPM, an efficient detector based on a decoupled two-stage design. It first extracts reusable class-agnostic visual representations, and then performs class-aware structural prompt matching with class names and cytomorphology prompts. On PentaCyto, CytoSPM outperforms existing methods in novel-category detection and open-vocabulary detection while maintaining efficient inference.