Enhanced deformable convolution improves image segmentation accuracy
Enhanced Deformable Convolution with Center-invariant Offset and Edge-aware Mask
Computer Vision and Pattern Recognition
Summary
Image segmentation is a way computers identify objects in pictures, but it can be tricky when the shapes are complex. The authors found that existing methods struggle to adapt shapes precisely and miss important details. They improved these techniques by using a method that focuses on the center of shapes and pays more attention to edges in images. This makes the computer better at recognizing and separating objects in pictures.
What this means in practice
- •For computer vision engineers: Enhance semantic segmentation models in applications like autonomous driving with improved spatial adaptation and focus on key image features.
- •For image recognition developers: Improve image classification accuracy by incorporating deformable convolutions with larger kernel sizes and edge-awareness in networks.
Authors
Yixiao Li, Xiaoyuan Yang, Jin Jiang, Minghao Zou, Guanghui Yue, Baoquan Zhao, Jun Liu, Wei Zhou
Abstract
Deformable convolution networks have recently become popular for many computer vision tasks, especially for semantic segmentation, because of their exceptional capabilities in dynamic spatial modeling. However, due to the dense deformable offsets and the lack of longer-range dependencies, they can not fully adopt proper and precise deformations for feature representations. To tackle the issues, in this paper, we propose Enhanced Deformable ConvNets (EDCN) for semantic segmentation. Specifically, a novel Enhanced Deformable Convolution (EDC) is exploited in the decoder, which integrates the Center-invariant Offset Module (COM) and Edge-aware Mask Module (EMM). The COM employs larger kernels and eliminates deformations at the kernel center, obtaining offsets that are more in line with the target from richer spatial information. Concurrently, the EMM obtains the significance of image content via Sobel edge detection, then selectively applies deformations based on the content significance, minimizing unnecessary deformations associated with relatively less important information, thereby avoiding impact from less informative regions. Experiments show that EDC outperforms state-of-the-art deformable convolution variants, including Deformable ConvNets V1-V4 and Entire Deformable ConvNets, across mainstream segmentation datasets with various decoder settings. Moreover, ablation studies confirm the effectiveness of each component. In addition, visualizations illustrate that EDC enhances spatial adaptation and target focus. We further analyze the extendibility of EDC to larger kernels on the image classification benchmark. Code will be publicly released.