DeltaS improves video memory management for streaming models
DeltaS: Reading the Gated Linear Attention State for KV Cache Eviction in Streaming Video
Computer Vision and Pattern RecognitionComputation and LanguageMachine Learning
Summary
When computers watch long videos and answer questions about them, they remember parts of the video in a special memory called a KV cache. But this memory can get full, so the computer has to decide what parts to forget before it even knows the question. The authors found a new way called DeltaS that looks at how much the memory changes over time to pick what to keep, without needing the question first. This method works better and faster than older ways, helping computers handle long videos more efficiently.
What this means in practice
- •For video streaming engineers: Manage memory in real-time video-language systems to retain the most informative video segments without knowing future queries.
- •For ai system developers: Optimize hybrid attention models for long video processing by improving key-value cache eviction strategies with low computational cost.
Authors
Taeyoun Kwon, Seungjin Kim, Hyeonyu Kim, Moon Hwan Kim
Abstract
Recent video-language models increasingly adopt hybrid architectures that interleave linear and full attention layers for efficient long-context processing. While the recurrent state of linear attention remains fixed in size, the KV cache of full attention continues to grow with the video stream, making eviction necessary under a bounded memory budget. The key challenge in streaming is that eviction must occur before the question arrives, so what to retain has to be decided without the question. Existing eviction methods derive token scores from the KV cache itself, using position, attention, or key-value representations, and attention-based scores further require proxy queries or extra computation. Hybrid backbones offer another source of signal. In gated-delta linear attention, the recurrent state is updated by the residual between each input and what can already be retrieved from the state, so its change over a chunk of frames reflects how much new information the chunk brings. We propose DeltaS, a query-agnostic, training-free method that retains video chunks inducing larger normalized state change, or state drift. In a controlled comparison with the budget and retention policy held fixed, state drift outperforms position-, attention-, and key-value-based signals. With a signal costing only 1.9% of the forward pass, DeltaS surpasses the strongest query-agnostic bounded-memory baseline by 2.1 points on average across six long-video benchmarks and by 5.6 points on the longest benchmark. These results suggest that the two memories of hybrid architectures can work cooperatively. Code is available at https://github.com/MaumAI-Company/DeltaS.