Towards Comprehensive Basketball Understanding
2026-08-24 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors created BasketballBench, a new set of basketball questions based on real NBA data, to test how well computers understand games by combining skills like recognizing players, actions, and game events. They also made BasketballSkills, a tool that uses different specialized methods together to answer these questions better than other models. Their tests show that current models struggle when multiple understanding skills are needed at once, but BasketballSkills does better by carefully combining specific basketball knowledge.
multimodal benchmarkplay-by-playrosterpossession-level clipMLLMperception toolsretrieval toolsdomain-specific capabilitiesNBA seasonstructured game knowledge
Authors
Yirong Hu, Jiayuan Rao, Yu Zhang, Shangzhe Di, Weidi Xie
Abstract
Understanding a basketball game requires recognizing events, localizing actions, identifying players, and relating these to structured game knowledge. Existing benchmarks primarily evaluate these abilities one at a time, leaving the interactions among these abilities under-explored. We introduce BasketballBench, a multimodal benchmark comprising 7,980 questions across ten tasks in text, image, and video. It is built from the 2025-2026 NBA season and includes official playby-play, rosters and profiles for 530 active players, and 2,501 possession-level broadcast clips. We further propose BasketballSkills, an agent that composes eight basketball-specific perception and retrieval tools under four reusable skills that specify tool order, evidence bindings, and stopping conditions. Experiments show that current MLLMs struggle particularly on questions requiring the integration of multiple capabilities, whereas BasketballSkills outperforms them, highlighting the effectiveness of explicitly composing domain-specific capabilities for comprehensive basketball understanding.