Video large language models get faster without retraining costs
Training-Free Inference Acceleration for Video Large Language Models: Methods, Trade-offs, and Evaluation
Computer Vision and Pattern Recognition
Summary
Processing long videos with large language models is very slow and uses a lot of computer memory. The authors review different ways to speed up these models during their use without needing extra training. They show how changing parts of the model’s steps affects both speed and accuracy, and why some improvements don't always make the whole process faster. They also compare past methods and run their own tests to find the best balance between speed and performance.
What this means in practice
- •For video processing engineers: Reduce computation time and memory use when running video large language models without retraining them.
- •For cloud service operators: Optimize cloud resources by applying inference acceleration methods for long video understanding tasks.
A survey. It maps existing work.
Authors
Yuqi Li, Chuanyue Yan, Neil Rodrigues, Xianglong Wang, Xiaoqin Feng, Junming Liu, Xi Xiao, Zhongwei Cheng, Wei Li, Lin Chen, Yingli Tian
Abstract
Video Large Language Models (VLMs) have demonstrated strong capabilities in video understanding and reasoning, but processing long visual sequences incurs substantial computational, latency, and memory overhead. This survey examines training-free inference acceleration, focusing on reducing inference costs while preserving task-relevant spatiotemporal evidence. We organize existing methods by their intervention stages in the inference pipeline, covering input, visual encoding, projector, prefill, and decoding. Post-training quantization (PTQ) is discussed separately as a cross-stage extension with varying calibration requirements. We analyze how intervention position affects evidence preservation and computational savings, and why local efficiency improvements may not translate into end-to-end acceleration. We complement this analysis with quantitative evidence from prior studies and our own experiments under a shared evaluation setting to assess accuracy-latency trade-offs.