Papers for

online video platforms

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Interaction-aligned method improves video segment detection for action-entity queries

IAE-VTG: Interaction-Aligned Action-Entity Video Temporal Grounding

Abstract: Video Temporal Grounding (VTG) localizes the video segment that matches a natural-language query. Many queries describe an action performed by a particular entity. Existing methods often encode the query as a whole or use general video-text interactions, without explicitly checking whether the action and entity occur together. They may therefore select a segment that contains both concepts but not the event described by the query. We propose Interaction Aligned Action-Entity Video Temporal Grounding (IAE-VTG), which models this rela?tionship at both the representation and training assignment levels. First, the Fine-grained Disentangled Interaction Module (FDIM) separates action and entity related query information and aligns it with complementary motion and appearance features. It then combines token-level interactions to build representations that capture the relationship between the action and entity. Second, Interaction-Sensitive Assignment (ISA) adds this interaction evidence to bipartite matching, so training targets are selected using both temporal overlap and semantic compatibility. This reduces supervision from temporally plausible but semantically incorrect proposals. Experiments on QVHighlights, Charades?STA, and TACoS show that IAE-VTG consistently improves strong baselines and achieves competitive or state-of-the-art performance on standard grounding metrics. Additional analyses show that the method is especially effective when similar actions or entities appear at multiple times and produces more reliable assignments for complex events.

Wed 9 SeptComputer Vision and Pattern RecognitionMultimedia
The gist
Finding the exact part of a video that shows a specific action happening with a certain object or person is hard because many methods only look for the action and object separately. The authors created a system called IAE-VTG that carefully checks whether the action and the entity actually happen together in the video. Their method breaks down the query into parts about the action and the entity, matches these with video features, and uses this combined understanding to better identify the correct video segment. Tests show it works better especially when the video shows similar actions or objects multiple times.
Open 2609.09736v1