Streamahead speeds up real-time video question answering with adaptive computing
StreamAhead: Real-Time Streaming Video Question Answering via Adaptive Precomputation
Distributed, Parallel, and Cluster ComputingComputer Vision and Pattern Recognition
Summary
Answering questions about live video fast is tough because computers must process lots of video frames quickly and be ready to answer questions at any moment. The authors introduced StreamAhead, a system that selectively prepares video parts at different detail levels, doing more work only when needed right before answering a question. This way, StreamAhead balances the workload, reducing delay while keeping up with video speed. It runs faster than previous methods and can handle high frame rates with quick responses.
What this means in practice
- •For video platform engineers: Integrate StreamAhead to deliver low-latency answers for live video streams while maintaining smooth video playback rates.
- •For security monitoring teams: Use StreamAhead to speed up automated questioning of surveillance footage in real time, improving incident response times.
Authors
Haoyang Fan, Dhruv Parikh, Rajgopal Kannan, Viktor Prasanna
Abstract
Real-time streaming video question answering with vision-language models must answer queries that arrive at any time with low latency while sustaining the video-processing rate required by the application. Existing systems construct key-value (KV) caches for the entire video as it arrives, although a query typically uses only a few of the preceding frames as evidence. This highlights a fundamental trade-off: performing more computation as video arrives reduces query-time computation but can increase processing lag, whereas deferring computation can increase the video-processing rate but requires additional query-time computation for retrieved content. We present StreamAhead, an algorithm-system co-design that exploits this trade-off through adaptive precomputation: dynamically controlling how much computation to complete for each video segment before a query arrives. StreamAhead encodes video segments independently and lets each remain at a different representation level, from inexpensive visual features to costly KV caches, completing KV caches only for the retrieved segments when a query arrives. StreamAhead profiles offline the cost of advancing a segment to each representation level. Its runtime uses these profiles to schedule segment computation concurrently with query execution, pipelining it across heterogeneous CPU and GPU resources while maintaining the required video-processing rate. StreamAhead achieves accuracy competitive with state-of-the-art methods with up to 10.4x lower time-to-first-token (TTFT), and up to 5.9x lower TTFT than state-of-the-art streaming systems. It sustains 30 FPS with P95 TTFT under 0.5 s, and up to 57 FPS with P95 TTFT under 2 s. To our knowledge, StreamAhead is the first real-time streaming video QA system with competitive accuracy, setting a new Pareto frontier between video-processing rate and TTFT.