Allocation Before Ranking: Decoupled Token Compression for OmniLLMs
2026-08-03 • Artificial Intelligence
Artificial IntelligenceSound
AI summaryⓘ
The authors argue that the common way of compressing tokens in OmniLLMs, which ranks all tokens together and keeps only the top ones, is flawed because it favors audio tokens over video. They propose Macer, a new method that sets separate token limits for audio and video and ranks tokens within each type separately. This approach reduces the number of tokens needed while keeping accuracy high across different audio and video tasks. Their results show Macer maintains almost full performance even when using only 25% of the tokens and improves efficiency compared to previous methods.
OmniLLMtoken compressiontop-K rankingmultimodal tokensaudio-video allocationMacer compressortoken retentionFLOPsbenchmark performance
Authors
Zhenghui Guo, Yilin Yang, Yuanbin Man, Miao Yin, Weidong Shi, Rabimba Karanjai, Omprakash Gnawali, Chengming Zhang
Abstract
Token compression in OmniLLMs is typically posed as a single saliency-ranking problem: score each multimodal token, keep the top-K. We argue this abstraction is mis-specified. The same attention score simultaneously decides two things: how much retained capacity each modality receives, and which tokens within a modality are kept. A shared top-K rule therefore inherits this audio-favoring allocation prior, spending retained capacity on audio before video tokens have a chance to compete. We propose Macer, a training-free compressor that first assigns explicit audio and video budgets, then performs allocation-normalized ranking within each modality at modality-specific shallow layers. Macer significantly reduces token cost while preserving accuracy across audio-grounded, audio--video joint, visual-dominant, and video-centric benchmarks. At 25 % retention, Macer preserves 98.7 % of full-token performance on Qwen2.5-Omni-7B and 97.3 % on Qwen2.5-Omni-3B. On Qwen2.5-Omni-7B, this 25 % setting reaches OmniZip-level performance at 45 % retention while using lower FLOPs. On OmniVinci-9B, the same allocation-before-ranking principle improves over shared top-K ranking by up to 12.9 points.