Automated analysis struggles with identity in amateur sports video
Prompt, Probe, Train, or Annotate? Single-camera sports video understanding in amateur settings
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Videos of school sports are often filmed with a single fixed camera and without professional quality, making it hard to track who is doing what in team games. The authors studied volleyball videos from amateur matches to see if various computer methods can accurately identify players and actions. They found that different approaches work well at different tasks like finding play segments or spotting ball contacts, but none can reliably identify players when their jersey numbers are hidden. This shows it's still hard for AI to fully understand chaotic, real-world sports footage from amateur settings.
What this means in practice
- •For sports analytics teams: Convert single-camera amateur volleyball footage into detailed play statistics using combined AI and manual methods tailored to the sport’s rules.
- •For sports video platform developers: Improve event segmentation and action spotting in single-camera sports videos using prompted vision-language models and lightweight trained specialists.$Commercial implications: Enables automated sports video editing tools that can handle amateur footage, creating products for amateur sports streaming services.
Authors
Sai Varun Kodathala, Prashanth Pollishetty, Jaylen Cargill
Abstract
Video understanding is usually benchmarked on curated, single-actor, or professionally filmed clips, and a strong score there is routinely read as evidence a model is robust enough for deployment. Amateur team sport is a useful, largely untested place to check that assumption: over eight million students played a school sport in the United States in 2024-25 alone, almost none of it filmed by more than a single fixed camera, with several candidate actors crowded into frame and no operator or second angle to fall back on. Using volleyball as a test case, we ask whether strong performance on general video and world-model benchmarks translates into reliable, per-player attribution once footage is this chaotic, turning footage into statistics through a chain of tasks from finding play boundaries to naming who did what. We evaluate four approaches (prompting and agentic reasoning over frontier vision-language models, classical computer vision with small trained specialists, self-supervised video world models, and manual annotation) at every stage, on 66 amateur matches with 46,648 human-labelled contacts, filmed under conditions no published benchmark uses. No single paradigm wins every stage, and static, single-frame computer vision is not competitive at any stage involving motion or identity. A prompted model segments matches well, yet a far smaller trained model beats it at spotting contacts for a fraction of the cost, and the sport's own rules recover rally outcomes the pixels cannot. Identity is where every automated approach struggles: a jersey number is a static fact temporal reasoning cannot recover if never visible, unlike sporting action, a repeated motor pattern a temporal model can exploit, which is why holistic reasoning improves event detection while identity stays unchanged. We close with where each approach earns its cost, and what transfers beyond volleyball to amateur sport.