DGSeg: Dynamic Gating of Semantic-Spatial Guided Predictions for Reasoning Segmentation
2026-07-06 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors study a way to better find specific objects in images based on complex language descriptions. They note that usual methods simplify the language clues into few hints like points or boxes, which can be unclear or noisy. Their method, DGSeg, uses a language model to create two different types of clues—one about what the object is and one about where it is—and processes these clues separately. Then, a smart system decides how to combine the two outputs to avoid mistakes. Tests show that their approach works better than earlier methods on tough image-language tasks.
Reasoning segmentationMultimodal Large Language ModelsVision-language reasoningSemantic cuesSpatial cuesSegmentation modelDynamic gatinggIoUReasonSeg dataset
Authors
Ruizhe Zeng, Siyu Cao, Lu Zhang, Zhiyong Liu
Abstract
Reasoning segmentation aims to predict pixel-wise masks for targets given complex language queries. Existing approaches leverage Multimodal Large Language Models (MLLMs) for vision-language reasoning and generate intermediate target cues (e.g., points or boxes) to guide a segmentation model. However, compressing rich reasoning into sparse cues often introduces ambiguity and noise, preventing these cues from accurately preserving the reasoning intent. While multiple complementary cues can enrich target information, existing methods typically feed them jointly into a single segmentation process, allowing ambiguous or erroneous cues to affect the entire prediction. Therefore, we propose DGSeg, a reasoning segmentation framework that learns to fuse predictions guided by semantic and spatial cues. Specifically, the MLLM jointly reasons about both target identity and spatial location, producing complementary semantic and spatial cues that are fed into separate segmentation branches. Their predictions are adaptively integrated by a lightweight dynamic gating module trained with relative branch-quality supervision to suppress noisy or conflicting regions. Extensive experiments demonstrate that DGSeg consistently outperforms strong baselines on multiple benchmarks and achieves 69.6% and 67.3% gIoU on the challenging ReasonSeg validation and test splits. Code is available at https://github.com/RZZeng/DGSeg.