Oriented object detection improves visual grounding in aerial images
A Unified Framework and Dataset for Oriented Object Visual Grounding in Remote Sensing
Computer Vision and Pattern Recognition
Summary
Finding objects in aerial photos based on descriptions is tricky when objects are rotated in different directions. Most existing methods use simple horizontal boxes that don’t fit these shapes well. The authors introduced a set of models called O2-VG that find objects using tilted boxes that match their true orientation. They also created a new dataset with pictures, descriptions, and rotated boxes to train and test their models. This helps computers understand and locate things in satellite images more accurately.
What this means in practice
- •For remote sensing analysts: Enable more precise detection of angled objects described by text in satellite or aerial images.
- •For geospatial intelligence teams: Support retrieval of specific oriented objects in large remote sensing image collections using natural language queries.
Authors
Zeyu Ding, Yong Zhou, Jiaqi Zhao, Wen-Liang Du, Xixi Li, Hancheng Zhu, Rui Yao, Abdulmotaleb El Saddik
Abstract
Visual grounding in remote sensing images aims to locate objects described by referring expressions. Most existing methods predict horizontal bounding boxes, which are often inaccurate for objects with arbitrary orientations. To address this limitation, we introduce O$^2$-VG, a family of models for oriented object visual grounding with three complementary designs. Specifically, O$^2$-VG-Trans is a cross-modality transformer for oriented object visual grounding. It establishes a strong discriminative foundation for the model family. Building upon it, O$^2$-VG-Uni predicts universal oriented proposals for possible foreground objects without specific text prompts. It also supports object retrieval through cached proposal embeddings. Using these universal oriented proposals as input prompts, O$^2$-VG-VLM is an autoregressive vision-language model. It generates oriented box token blocks in parallel through multi-token prediction. In addition, we construct DIOR-R-RSVG, a dataset for oriented object visual grounding in remote sensing images. It provides image, expression, and oriented box triplets for training and evaluation. Together, the O$^2$-VG family provides a flexible framework that spans discriminative transformers and generative vision-language models. It achieves superior performance across multiple benchmarks. Code is available at https://github.com/wokaikaixinxin/ai4rs.