CRAFT: Compression via Recursive Adaptive Fusion of Video Tokens for Vision-Language Models
2026-08-03 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors propose a new method called CRAFT to reduce the number of video tokens that vision-language models need to process, which helps save computing power and memory. CRAFT smartly merges similar tokens using a combination of fixed rules and learned fusion techniques, keeping important details intact. This approach maintains the original position information of tokens and works smoothly with existing language models. Their experiments show that CRAFT can compress tokens by about 8 times with only a small drop in accuracy, improving efficiency compared to previous methods.
vision-language modelsvideo tokenstoken compressionspatio-temporal redundancyadaptive fusionposition-aware weightingchannel-wise gatinglinear combinationpre-trained language modelscomputational efficiency
Authors
Yu Chen, Xiaohong Li, Xiaole Wang, Jianjin Zhang, Jun Sun, Yafeng Deng
Abstract
In video understanding, vision-language models (VLMs) must ingest massive numbers of visual tokens, causing the computational and memory cost of the prefill stage to rise sharply. Such visual sequences are highly redundant along the spatio-temporal dimension, yet a high compression ratio is often accompanied by the loss of critical details. Existing token-compression methods either employ heuristic, training-free compression with limited content adaptivity or introduce additional modules that require expensive alignment training, leaving the trade-off between efficiency and adaptivity unresolved. To alleviate this limitation, we propose CRAFT: Compression via Recursive Adaptive Fusion of Video Tokens. CRAFT recursively merges tokens by decoupling parameter-free token selection from learnable token fusion: global similarity determines which tokens to merge, while a position-aware weighting module and a content-adaptive channel-wise gate learn how to fuse them. The whole compression pipeline is query-agnostic. Because every retained token is a linear combination of the original tokens, CRAFT preserves their true spatio-temporal coordinates and stays aligned with the pre-trained language model's input distribution. Experiments on multiple representative video benchmarks show that CRAFT consistently outperforms prior state-of-the-art token-compression methods. At about $8\times$ compression, it retains roughly $97\%$ of the backbone's average accuracy and shows significant efficiency improvement.