Papers for

video security monitoring teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Simple method improves event recognition across audio and video

SIMPLE: Semantic Structure-Aligned Prompt Learning for Open-Vocabulary Audio-Visual Event Localization

Abstract: Open-vocabulary audio-visual event localization aims to infer explicit segment-level categories for both seen and unseen audio-visual events. Existing solutions rely heavily on hand-crafted prompt templates, whose predictions are highly sensitive to subtle wording variations. Learnable prompts remove this manual tuning burden, but when trained solely on seen events, they tend to overfit to seen categories and distort the inter-event semantic topology required for generalization to unseen categories, a failure we term semantic topology drift. In this paper, we propose SIMPLE, a semantic structure-aligned prompt learning framework that aligns cross-category semantic topology between learnable and hand-crafted prompts across asynchronous audio and visual streams. Specifically, our SIMPLE combines Learnable Continuous Prompt Tuning (LCPT) with Semantic Topology Consistency Regularization (STCR). The LCPT automates prompt engineering through data-driven optimization, while the STCR module aligns the category-wise similarity distributions of learnable and hand-crafted prompts under random category subset sampling to alleviate semantic topology drift. Comprehensive experiments and analyses on the OV-AVEBench demonstrate that our SIMPLE framework achieves state-of-the-art performance on the seen and unseen splits, while using roughly half the trainable parameters and roughly one-third the training time.

Sun 11 OctMultimediaSound
The gist
Recognizing specific events from combined sound and video, especially ones not seen before, is hard because current methods rely on fixed word prompts that can be tricky to get right. The authors present SIMPLE, a new approach that learns these prompts automatically while keeping their meaning consistent across categories, reducing confusion when encountering new events. Their method improves accuracy on both known and new events while being faster and using fewer resources than previous techniques.
Open → 2610.15342v1