Agreement-Based Audio-Visual Segmentation:Champion Report for the MeViS-Audio Track in the 8th LSVOS Challenge
2026-08-10 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors created a system that finds objects mentioned in spoken descriptions during a video and returns empty results if the object isn't there. Their method first changes speech to text, then uses several models to track objects in the video. Instead of relying on one guess, they pick the tracking result that agrees best with others. They also use some rules to handle more complex speech, like counting objects or directions. Finally, their system decides if the described object is present by combining different scores, and their approach ranked first in the challenge.
speech-to-textvideo segmentationobject trackingmask agreementmotion expressionsaudio-visual classificationplural rulesvideo-level classifierMeViS-Audio challenge
Authors
Yiwen Ren, Jianing Liu, Yingxin Wang, Kexin Zhang, Licheng Jiao, Lingling Li, Xu Liu
Abstract
The MeViS-Audio track asks a system to segment the objects described by a spoken motion expression throughout a video and to return empty masks when the described target is absent. We present a simple staged solution. Qwen3-ASR first converts speech into text. Several video mask tracks are then produced with complementary grounding and segmentation models. Instead of trusting a single prediction, we select the track that has the highest average mask agreement with the other candidates. A small set of explicit direction, count, and plural rules corrects queries that require more than ordinary single-object tracking. Finally, a video-level classifier combines visual, audio-visual, and within-video query scores to decide whether any target is present. The submitted system obtains 0.5952 J &F, 0.7931 no-target accuracy, 0.9205 target accuracy, and a final score of 0.769589. The challenge organizers notified our team that this result ranked first in the track.