Visual token compression preserves image details in vision models
CoViST: Visual Token Compression via Composable States
Artificial Intelligence
Summary
Vision-language AI models analyze images by breaking them into many small pieces called tokens, which can be slow to process. The authors found that common ways to shrink these tokens lose track of where in the image they came from and how important each token is. They created CoViST, a method that keeps information about each token’s position and contribution even after compression, making the models faster without losing accuracy. Their tests show CoViST keeps nearly all the original model’s performance while using far fewer tokens.
What this means in practice
- •For ai product developers: Run vision-language models more efficiently by compressing image tokens with preserved spatial and contribution information to maintain accuracy at lower computation costs.
- •For robotic vision engineers: Improve real-time image understanding in robots by using CoViST to reduce processing time without losing key visual details needed for decision making.
Authors
Qi Zhang, Xiandong Meng, Ronggang Wang, Siwei Ma
Abstract
Visual token compression lowers the inference cost of vision--language models by representing images with fewer tokens. However, most existing methods compress visual tokens to a reduced set, leaving the amount of visual evidence represented by each token and its original spatial context implicit. Therefore, the compressed representation does not explicitly encode how much visual information each representative carries or where it lies in the original image. This limitation arises even after a single reduction and becomes more pronounced when compression is repeated across decoder layers. To address this issue, we propose CoViST, a training-free framework that represents a compressed image as a composable visual state. Specifically, the state combines representative features with original positions, effective contribution weights, and reusable selection metadata. CoViST constructs this state through coverage-guided selection and conservation-based contribution composition, and explicitly incorporates its contribution and positional information into decoder attention. Each component of the state retains its interpretation under successive reductions, enabling the same formulation to support both fixed compression before prefill and progressive compression within the decoder. Experimental results on seven LLaVA-1.5-7B benchmarks show that CoViST-Fixed retains 99.9\%, 99.5\%, and 98.1\% of uncompressed performance at 192, 128, and 64 tokens, respectively, and CoViST-Pro retains 99.8\%, 99.9\%, and 99.1\% at the corresponding layer-average budgets, outperforming state-of-the-art methods under their respective budget settings. Code will be released publicly.