Kairos dataset captures detailed video events over long times
Kairos: A Dataset for Fine-Grained Video-Language Modeling over Space, Time, and Dynamics
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Many video models only look at short clips and miss how things change over time. The authors created Kairos, a collection of long videos with detailed labels that show what’s happening moment by moment. These labels include actions, appearances, and interactions, helping computers understand videos more deeply. Kairos lets models learn continuous changes and contextual details over a long time rather than just snapshots.
What this means in practice
- •For video app developers: Create apps that analyze ongoing actions and changes in long videos beyond brief clips using detailed time-aligned annotations
- •For game designers: Build games with realistic visual event progression by training AI on datasets showing continuous actions and interactions over time
Authors
Ruibo Ming, Lei Sun, Deheng Zhang, He Zhang, Jialu Li, Jian Wang, Zhendong Li, Mengshun Hu, Danda Pani Paudel, Luc Van Gool, Jinjin Gu
Abstract
Many emerging video language modeling tasks require systems to move beyond clip-level abstraction and model visual content as it unfolds over extended time horizons. However, most existing video datasets rely on coarse or sparsely aligned supervision, which compresses temporal variation and limits the ability of models to learn reusable representations of continuous visual dynamics. We introduce Kairos, a video dataset for video-language modeling with time-resolved annotations. Kairos consists of long-duration videos, ranging from ten minutes to half an hour, annotated with fine-grained temporal alignment. The annotations capture ongoing actions, entity appearances and attributes, interactions, and evolving contextual cues along the video timeline. This time-resolved structure supports fine-grained evaluation, long-range modeling and reasoning, instruction data construction, representation learning, and video generation. Kairos provides a general-purpose foundation for modeling visual experiences over time.