QCell: Recombining and Aligning Cell Queries for Overlapping Instance Segmentation

Computer Vision and Pattern RecognitionArtificial IntelligenceMachine Learning

Summary

The authors address the problem of identifying individual overlapping cells in microscope images, which is hard because cells can be partly see-through and blend together. They developed QCell, a new method that uses a kind of query system to separate overlapping cells by combining parts of cell representations and learning to tell cells apart better. They also created a new dataset to test their method. Their approach performs better than existing methods on standard tests for cell segmentation.

Authors

Yaroslav Prytula, Anton Popov, Dmytro Fishman

Abstract

Instance segmentation of overlapping cells in microscopy remains challenging due to semi-transparent structures that produce weak boundaries and mixed visual evidence in overlap regions. Existing methods address this through local regions of interest or shape priors but lack global reasoning across overlapping objects. We present QCell, a novel query-based model that de-overlaps cell instances in microscopy scenes. Our approach combines (i) an instance recombination module that decomposes and recombines query representations in latent space, enabling the model to reason about complete object structure under overlap, and (ii) a contrastive query alignment objective that combines distinctive instance feature learning and separation of overlapping cell queries. We additionally introduce a new Organoid dataset benchmark for overlapping cell segmentation. We show that QCell outperforms state-of-the-art methods across multiple benchmarks, achieving +2.2 AP and +2.7 AJI on ISBI2014. Code is available at https://github.com/SlavkoPrytula/QCell