ET-Prune: Evidence-Aware Dynamic Budgeting for Visual Token Pruning in Text-Rich MLLMs
2026-08-03 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionComputation and Language
AI summaryⓘ
The authors developed ET-Prune, a method that reduces the number of visual tokens a multimodal language model uses during inference, especially for tasks involving text recognition (OCR). Instead of cutting tokens randomly, ET-Prune smartly keeps important text-related parts based on the question being asked, ensuring crucial information isn't lost. Their approach adjusts how many tokens to keep depending on how scattered or dense the text evidence is, improving performance while using fewer tokens. Tests showed ET-Prune performs better or equal to other pruning methods on multiple models and benchmarks with roughly half the tokens, balancing speed and accuracy effectively.
Visual token pruningMultimodal large language modelsOCR (Optical Character Recognition)Evidence allocationQuestion-conditioned evidenceToken budgetInference costDynamic token pruningVisual tokensMultimodal benchmarks
Authors
Zizhong Ding, Junxian Li, Kai Liu, Shaoqiu Zhang, Xiao Xiao, Linghe Kong, Yulun Zhang
Abstract
Visual token pruning reduces the inference cost of multimodal large language models, but a fixed token ratio is poorly matched to text-rich inputs. In OCR-centric tasks, decisive evidence can be a small number, label, or field whose relevance is specified by the question; indiscriminate pruning can erase that evidence while retaining visually salient but irrelevant regions. We present ET-Prune, a training-free framework that casts pruning as evidence allocation. It derives question-conditioned evidence from a decoder-side partial query-key block, safeguards text-like spatial regions, and converts evidence uncertainty and density into a sample-specific token floor. Three progressive middle-layer events then move the sequence toward this budget, retaining more tokens for diffuse or text-dense evidence and pruning concentrated evidence more aggressively. At the observed point estimates from one deterministic pass per configuration, ET-Prune leads or ties among pruned methods in all six backbone-benchmark comparisons at roughly half tokens. On OCRBench-v2, it leads the strongest pruned baselines by 1.80 and 0.68 percentage points on Qwen3-VL-8B and InternVL3.5-8B, respectively, while retaining about half of the visual tokens; on MMBench v1.1, it reaches 0.8467 circular exact-matching accuracy versus 0.8437 for Vanilla at 54.45% average visual-token retention. These results show a favorable observed quality-cost trade-off for evidence-aware dynamic budgeting in text-rich multimodal inference.