Papers for

software developers in computer vision

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Segbanana improves medical image segmentation using visual knowledge and editing

SegBanana: Steering Unified Multimodal Models into Medical Segmenters

Abstract: Medical image segmentation remains challenging in practical deployment, as models often struggle to generalize beyond the distributions covered by their training data and high-quality pixel-level annotations are typically unavailable for adaptation. Inspired by the cross-task transferability of large language models, we investigate whether unified multimodal models (UMMs) can transfer their pretrained visual understanding, reasoning, and generation capabilities to medical image segmentation without task-specific post-training. By recasting segmentation as structured visual generation, we find that frontier UMMs (e.g., Nano Banana) already exhibit basic segmentation capabilities across diverse clinical scenarios, but still struggle with challenging tasks requiring specialized anatomical or domain-specific knowledge. We further show that these limitations can be effectively mitigated by incorporating visual anatomical knowledge from in-context exemplars, expanding candidate solutions through repeated sampling, and refining suboptimal predictions via targeted editing.Motivated by these observations, we propose SegBanana, to our knowledge, the first agentic visual generation framework for training-free medical image segmentation. SegBanana builds on a frozen UMM as the core generative model, augmented with Anatomy-Aware Knowledge Retrieval and Comparative Quality Critique to unlock its potential segmentation capability. A State-Aware Multimodal Controller maintains structured state and iteratively orchestrates these tools, repeatedly refining intermediate predictions toward higher-quality masks. Across eight medical segmentation datasets, SegBanana achieves an average mDice of 77.45%, outperforming representative generalist (SAM3 and SegGPT) and medical-specific (BiomedParse and MedSAM3) baselines by at least 14.93 points, while remaining robust to out-of-domain visual supports.

Mon 28 SeptComputer Vision and Pattern Recognition
The gist
Medical image segmentation, which means marking important parts of medical pictures, is hard because models often can’t handle unseen cases well, and detailed training data is scarce. The authors found that unified multimodal models, which understand and generate images and text, can do simple segmentations but struggle with complex medical details. They created SegBanana, which uses a frozen multimodal model combined with visual knowledge and iterative editing to improve segmentations without extra training. SegBanana performed much better than several existing general and medical segmentation methods on multiple datasets.
Open → 2609.34235v1