Adaptive token evaluation boosts image reconstruction with less computation
Selective Amortization of Full-Budget Counterfactual Reasoning for Visual Token Communication
Artificial Intelligence
Summary
Sending images as small pieces called tokens can save data, but picking which tokens to send needs a lot of computing. The authors created a method called ACV-Gate that learns to guess the value of many tokens quickly and only does careful checks on the most important ones. This approach reduces the computing needed while keeping image quality high, especially when sending images with very limited data. Tests show it works well on several image datasets and sizes.
What this means in practice
- •For image compression developers: Improve encoder efficiency by selectively evaluating token candidates to enhance quality under strict bit rate limits.
- •For streaming platform engineers: Integrate selective token evaluation to reduce server-side encoding costs while maintaining image reconstruction quality for low bandwidth users.
Authors
Qinglei Qi, Zhihe Liang, Fengzhan Jing, Shenao Zhu, Lei Zhang, Chenyang Zhang, Shuqing He, Jia Guo
Abstract
Generative image communication transmits compact semantic tokens under a limited packet budget, where token selection directly affects the final reconstruction quality after the complete packet is decoded. However, accurately estimating the terminal value of every candidate token requires repeated receiver-side reconstruction, resulting in substantial encoder-side computation. To address this problem, we propose ACV-Gate, an adaptive candidate evaluation framework that learns to approximate full-budget counterfactual evaluation and selectively assigns exact evaluations to the most informative candidates. Specifically, a set-aware student is trained using terminal advantages and regrets to predict candidate rankings directly, while a selective refinement mechanism evaluates only a bounded candidate set containing both Local-MDL and direct actions; cost-based thresholds further enable explicit control of the average evaluation workload. Experiments on CIFAR-10 show that ACV-Gate consistently improves reconstruction quality while substantially reducing candidate evaluations; at 0.20 bpp, the primary adaptive configuration improves PSNR over LocalMDL by 0.636 dB with only 2.13 candidate evaluations per image, corresponding to 27.60% of the calls required by the Exact-Full expert. Matched-candidate comparisons, synchronized GPU measurements, and evaluations on STL-10 and 384 *384 scale transfer further demonstrate consistent quality computation trade-offs, with particularly pronounced gains at low bit rates. These results show that combining terminal-value learning with selective candidate evaluation provides an effective and controllable mechanism for allocating encoder computation in packet-constrained generative image communication.