Method for flexible mix of image annotations improves instance segmentation
MixSIS: Mixed-supervised instance segmentation
Computer Vision and Pattern Recognition
Summary
Labeling all objects in images with detailed shapes takes a lot of time. The authors studied how mixing simpler labels like points or boxes with some detailed masks can still help computers learn to find and outline objects accurately. They created a system called AnnsMixer that turns these mixed labels into accurate object shapes, even when different objects use different kinds of labels in the same image. This method can save a lot of annotation work or improve the quality of object detection depending on how labels are assigned.
What this means in practice
- •For computer vision engineers: Reduce manual labeling time for training instance segmentation models by mixing different annotation levels based on object difficulty.
- •For data annotation teams: Enable annotation workflows that use simpler labels for easy objects and detailed labels for hard ones, improving overall labeling efficiency.
Authors
Christian Wilms, Emre Gülsoylu, Sven Schreiber, Simone Frintrop
Abstract
Weakly-supervised instance segmentation (WSIS) based on either box-, point-, or image-level annotations has been proposed to overcome the labor-intensive annotation process in fully-supervised instance segmentation. Despite decent results, most WSIS methods only focus on a single annotation level, ignoring that some objects are easy to spot from point annotations, while others require box or even mask annotations. Therefore, we propose the Mixed-Supervised Instance Segmentation (MixSIS) task, which allows a flexible mix of annotation levels (image, point, box, and mask) within a dataset. MixSIS has a high practical relevance, as it allows optimizing the annotation process by annotating challenging objects in more detail, while saving time on simpler instances with less detailed annotations. To address MixSIS, we propose AnnotationsMixer (AnnsMixer) as a baseline. AnnsMixer is a truly mixed-supervised instance segmentation system, which generates pseudo mask annotations from all annotation levels, even mixed within a single image. We evaluate AnnsMixer on both WSIS and MixSIS tasks, largely outperforming state-of-the-art methods. In three case studies, we also demonstrate how the unique features of MixSIS and AnnsMixer can substantially reduce the annotation time (-90%) or improve the results (+34%) in instance segmentation with a simple annotation rule or leveraging heterogeneous data. Our code is available at https://github.com/chwilms/MixSIS