Semantic grounding transforms test-time discovery into interpretable concepts

Relational Semantic Grounding for Test-Time Discovery

Artificial Intelligence

Summary

Sometimes machines see new things they weren’t trained on and only group them without naming them. The paper’s authors created a method that uses language models to describe known objects with meaningful features, then helps the system give names to new categories when they appear. This means the machine can explain new things it finds in words we understand, instead of just saying 'new group one' or 'new group two.' Their approach improves how well machines identify and explain things in the real world after they start working.

What this means in practice

Authors

Tianle Liu, Gongbo Cheng, Chenggong Ni, Minming Gu, Tao Zhou, Fuyuan Hu, Fan Lyu

Abstract

Test-Time Discovery (TTD) aims to recognize known classes and detect novel ones from a continuous unlabeled test stream after deployment. Existing TTD methods mainly rely on visual clustering or prototype comparison, and can only assign anonymous labels to discovered classes, leaving them difficult to interpret in open-world applications. In this paper, we propose SEGA, a Semantic Grounding with Attributes framework for TTD. SEGA leverages large language models to generate attribute-rich descriptions for known classes and constructs an attribute-anchored visual-semantic space during training. At test time, we introduce Relational Semantic Grounding (RSG), which grounds unknown samples with similar and dissimilar known-class anchors, prompts the LLM to generate candidate concepts, and reranks them in the learned semantic space. The selected concept is further used for confidence-guided test-time semantic stabilization. By coupling attribute-level semantic alignment with relational concept grounding, SEGA transforms anonymous test-time clusters into interpretable semantic concepts. Experiments show that SEGA improves TTD baselines with the promise of semantic grounding for open-world.