Woma enables fast AI detection and analysis in endoscopy videos
woma: a real-time foundation model and its fine-tuned models for endoscopy
Software EngineeringComputer Vision and Pattern RecognitionMachine Learning
Summary
Looking inside the digestive system is tricky, and doctors need help spotting important signs during endoscopy. The authors built woma, a smart computer program that learns from many unlabeled images to recognize features like polyps and lesions in real time. It can label parts of the colon, highlight possible disease areas, and works quickly on standard computers without extra tools. This helps doctors by providing accurate, fast support during procedures without needing huge computing resources.
What this means in practice
- •For hospital endoscopy teams: Run fast, accurate polyp detection and digestive tract analysis during live colonoscopy and gastroscopy procedures on standard GPU setups.
- •For medical device developers: Integrate real-time, label-free trained AI models for digestive imaging that do not require external AI frameworks or vendor dependencies.$Commercial implications: Supports creation of embedded diagnostic tools for endoscopy devices enhancing detection and classification, enabling new medical products.
Authors
Thang Tran, Lan Dang
Abstract
woma is a real-time foundation model for gastrointestinal endoscopy: a network trained without labels on about a million endoscopy frames, from which task models are fine-tuned. We contribute a systematic design for production. Requirements and pass marks were fixed before any run, eight candidates screened under pre-registered rules, self-supervised training taken to a stopping rule, then fine-tuning and deployment optimisation, all on one self-contained library, numbat. We also contribute woma itself with two fine-tuned models, every outcome reported met or missed. Our colonoscopy model finds and outlines polyps, names which colon segment is in view, suggests polyp type and grades bowel preparation. Our gastroscopy model names a station out of 22 protocol sites, flags and outlines lesions, and names one of seven findings. Every number was read on data never seen in training, and shipped weights were chosen on that record. In colonoscopy, 96% of polyps in a six-hospital PolypGen set are found at precision >=0.85, and 19 of 19 polyps across fifteen full REAL-Colon videos at 1.6 false alarms per procedure. In gastroscopy, landmark region is named correctly on 92% of frames from unseen patients, and 37 of 39 held-out neoplasia frames are flagged at specificity 0.91. On one workstation GPU every task runs over 1080p video at about 100 frames per second, faster than PyTorch, ONNX Runtime and TensorRT in all four precision regimes tested. TensorRT comes closest: one pass of our foundation model takes it 3 to 27% longer than ours, and we deliver 6 to 31% more frames per second from frame to results. A second build links no vendor library at all -- our own kernels over Vulkan -- so a site deploys two files and needs no toolkit, no cuDNN and no framework; in f32 it beats the CUDA build on the same card.