Zero shot model spots and ranks soccer events using video and commentary
Zero-Shot Multimodal Event Spotting in Soccer Broadcasts
Computer Vision and Pattern RecognitionMachine LearningMultimedia
Summary
Finding key moments during a soccer match automatically helps create highlights and analyze games. The authors present a method that detects and identifies events in entire soccer matches without needing special training for soccer or lots of example data. Their system combines short video summaries with match commentary and uses a general AI model to find when events happen. It performs well compared to existing models and runs faster by using compact video representations. They also added a step that ranks exciting moments with explanations to pick the best highlights.
What this means in practice
- •For sports video platform developers: Automatically detect and classify soccer game events in full broadcasts without custom model training, enabling highlight creation and content indexing.$Commercial implications: Allows building automated highlight generation products for broadcasters and streaming services without costly soccer-specific data or retraining.
- •For video archive managers: Index and search large collections of sports videos by events using a fast, zero-shot multimodal detection approach to save processing time.
Authors
Sadia Kamal, Mahmudul Hasan, Tony Braskich
Abstract
Automatic soccer event spotting supports highlight generation, video indexing, and sports analytics, but existing methods depend on large annotated datasets and soccer-specific training. We present a zero-shot framework that detects, classifies, and temporally localizes events in full-length soccer broadcasts without task-specific parameter training. Each match is segmented into overlapping clips represented by a compact visual storyboard and the aligned commentary transcript, which a general-purpose multimodal model processes jointly. On three manually annotated matches containing 130 events, the best configuration achieves 72.0% F1@5s and outperforms SoccerNet-trained CALF and NetVLAD++ in a cross-dataset setting, while the storyboard representation reduces inference cost by 72% compared with individual frames. We further analyze the effects of model family, input representation, and prompt optimization, and introduce an evidence-grounded excitement-scoring stage that ranks detected events with natural-language explanations for highlight selection.