TORINO: Token Reduction via Interpretable Concept Overlap in Vision-Language Models
2026-07-06 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors present TORINO, a method to reduce the number of visual tokens processed by vision-language models without retraining them. TORINO uses Sparse Autoencoders to find meaningful groups of visual tokens based on shared concepts, then merges or prunes tokens within these groups to remove redundant information. This approach dynamically adjusts how many tokens to keep based on the complexity of each image. Their experiments show TORINO can make models more efficient while keeping accuracy mostly intact.
Vision-Language ModelsVisual TokensToken ReductionSparse AutoencodersLatent SpaceSemantic RepresentationToken PruningToken MergingConcept OverlapEfficiency-Accuracy Trade-off
Authors
Riccardo Renzulli, Gabriele Spadaro, Shruthi Gowda, Alaa Eddine Mazouz, Van-Tam Nguyen
Abstract
Vision-Language Models (VLMs) have demonstrated impressive capabilities across different tasks, but their computational cost is dominated by the large number of visual tokens fed to the language model. Existing token reduction methods rely on attention-based scores or pairwise similarity, without an explicit semantic representation of each token. We introduce TORINO (TOken Reduction via Interpretable coNcept Overlap), a plug-and-play framework for adaptive visual token reduction in VLMs that requires no fine-tuning of the underlying model. TORINO leverages Sparse Autoencoders (SAEs) to project visual tokens into an interpretable latent space where token relationships can be analyzed through shared concept activations. Specifically, we define concept overlap as the degree of agreement between active SAE latents and use it to group tokens that share semantic content. Reduction within each group is then performed by either pruning or merging, providing a unified framework that preserves semantically important visual information while removing redundancy. Unlike fixed-budget approaches, TORINO dynamically adapts the reduction rate to input complexity, allowing different images to retain different numbers of tokens. Experiments across multiple vision-language benchmarks show that TORINO achieves favorable efficiency-accuracy trade-offs, reducing the number of visual tokens with minimal performance loss.