InterHier improves object detection with learned semantic hierarchy context
InterHier: Learning Interconnected Hierarchical Semantics for Open-Vocabulary Object Detection
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Detecting objects involves understanding categories, including new ones the system hasn’t seen before. Existing methods link categories together using fixed connectors, but these don’t always capture the true relationships well. The authors propose InterHier, which learns a flexible context to better represent category relationships in a hierarchy. This approach helps the system understand new categories more accurately and works well with current open-vocabulary object detectors.
What this means in practice
- •For computer vision engineers: Integrate InterHier to improve detection of new object categories in open-vocabulary detection systems.
- •For robotics developers: Enhance robot perception to recognize and classify unfamiliar objects by using learnable hierarchical semantic contexts.
Authors
Yeong-Jin Kim, Ho-Joong Kim, Seong-Whan Lee
Abstract
In this paper, we investigate the limitations of fixed, hand-crafted connectors in hierarchical semantic representations for open-vocabulary object detection. Existing methods establish semantic relationships between base categories and unseen novel categories by placing a fixed connector between adjacent super-/sub-categories. However, such fixed connectors may not optimally capture the relationships within a semantic hierarchy. To address this limitation, we propose interconnected hierarchical semantic representations (InterHier), which utilize a prepended learnable context to globally guide the interpretation of prompts containing hierarchical relationships. InterHier operates in two main stages. First, it constructs a hierarchy-aware prompt by integrating super-/sub-categories and prepending a learnable context. Second, it optimizes this learnable context to align visual region embeddings and textual embeddings. InterHier consistently improves performance over methods that rely on fixed connectors and can be seamlessly integrated into existing open-vocabulary object detection models. Experiments on open-vocabulary object detection benchmarks demonstrate that InterHier achieves competitive performance against state-of-the-art methods.