Colonoscopy polyp tracking improves without manual bounding box labels
Annotation-Free Representation Learning for Polyp Retrieval in Colonoscopy Videos
Computer Vision and Pattern Recognition
Summary
Colonoscopy videos show polyps from different angles, helping doctors understand them better. But to find and track the same polyp in many frames, computers usually need human-drawn boxes around polyps, which is slow and costly. The authors developed a method that learns to find and follow polyps over time using raw videos alone, with no need for these labels. This approach works better than existing methods and can find more images of the same polyp even from just a few examples.
What this means in practice
- •For medical imaging software developers: Build colonoscopy tools that identify and track polyps without needing manually labeled data, speeding up clinical workflow.$Commercial implications: Enables development of improved colonoscopy image tracking software sold to healthcare providers, reducing annotation costs.
- •For clinical it teams: Deploy and adapt polyp retrieval models on-site using only unlabeled procedure videos, facilitating local model updates without expert annotations.
Authors
Luca Parolari, Lamberto Ballan, Carlo Biffi, Loic Le Folgoc
Abstract
Colonoscopy is inherently comparative: reliable visual assessment of polyps often depends on complementary views acquired under different viewpoints and imaging conditions. Retrieving such views requires polyp tracklet representations, which recent contrastive methods learn from temporal cues but still build from manually annotated bounding boxes. We propose an annotation-free framework that constructs tracklet views directly from raw colonoscopy videos, enabling temporal self-supervised learning on full-procedure videos, including frames without visible polyps. On REAL-Colon, our method achieves state-of-the-art results on tracklet retrieval and re-identification, outperforming both self-supervised and supervised baselines. We then introduce few-shot polyp retrieval, a task closer to clinical use, where a few observations of a target polyp are used to retrieve its remaining tracklets: with only two examples, our method surpasses most competing methods given ten. Finally, we show that a modest number of unlabeled procedures suffices for competitive performance, making on-site model development more practical. Code is available at https://github.com/lparolari/freerep.