End-to-end method improves cell detection by combining images and cell relationships
End-to-End Cell Detection via Instance-aware Graph Modeling
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Detecting and identifying different types of cells in medical images is important for diagnosing diseases and planning treatments. The authors created a new approach that looks not only at how cells appear but also how they relate to each other in the tissue. Their method builds a graph that links cells based on their features and positions, then uses this graph to improve detection and classification in one integrated step. Tests on several kinds of medical images showed their method works better than previous ones.
What this means in practice
- •For medical imaging teams: Enhance automated analysis tools to detect and classify cells more accurately by jointly modeling cell appearance and spatial relationships.
- •For digital pathology labs: Improve diagnostic workflows by integrating end-to-end detection and classification algorithms that adapt to different staining protocols.
Authors
Ruochen Liu, Yalin Zheng, Jingxin Liu, Jianfeng Zhang, Shoujun Huang, Dexing Kong, Haofeng Li, Wei Lou
Abstract
Accurate cell detection and classification are crucial for pathological analysis, directly affecting diagnostic accuracy and treatment planning. To capture complex cellular interactions beyond visual appearance within the tumor microenvironment, several approaches have employed graph neural networks to model spatial and relational patterns among cell nuclei, yielding promising results. However, these methods typically adopt a two-stage paradigm of visual extraction followed by relational modeling, which necessitates separate tuning for each stage, thereby increasing pipeline complexity and hindering end-to-end joint optimization. In this paper, we propose an end-to-end framework for cell detection and classification that jointly models patch-level visual representations and instance-level interactions, which incorporates a dynamic graph construction module and an instance-aware graph network. Specifically, the graph construction module dynamically builds the graph structure using learnable queries derived from patch-level features as cell instance representations, with adjacency defined by integrating feature similarity and spatial distances. The instance-aware graph network performs adaptive instance filtering and feature reorganization, aggregating them over the cell graph into a topological latent state for a selective state-space transition driven by visual cues, fusing appearance and relational evidence. When evaluated on multiple datasets with different staining protocols for cell and nucleus detection, our method significantly outperforms existing approaches in both detection and classification performance. The code will be released at https://github.com/RuochenLiu23/IGM.