Coding agents autonomously compress large language models effectively
CompressionBench: How can your coding agents compress LLMs?
Artificial Intelligence
Summary
The paper looks at how AI coding assistants can shrink big language models while keeping them working well. The authors set up a challenge where an AI agent tries to reduce a model’s size within limits but still produce accurate results. They found that many agents could make smaller models that mostly kept the original’s quality, judged by tests on how well the compressed model predicts text and performs tasks. This helps show steps toward AI systems improving themselves by managing and refining their own models automatically.
What this means in practice
- •For machine learning engineers: Enable automatic reduction of pretrained model sizes to save storage and improve efficiency without major performance loss.
- •For ai infrastructure teams: Use agent-guided model compression to manage resource budgets dynamically in deploying large language models.
Authors
Xinrui Zhong, Zhifeng Li, Zhiming Wang, Yongqi Liu, Shanglin Yang, Yang Zhou, Zikun Li, Zhuoming Chen, Beidi Chen
Abstract
Recursive self-improvement requires AI systems to improve the models and infrastructure used by subsequent systems. LLM compression offers a concrete efficiency objective: retain a pretrained model's behavior within a smaller model storage budget. We study whether coding agents can carry out this step autonomously. We formulate compression as a terminal task in which an agent receives a source checkpoint, storage limit, inference environment, and time budget, then constructs compressed weights and executable inference code. The agent controls its algorithm and experiments; perplexity feedback guides search, and independent verification checks the final model. Across six agents and six model--size settings, 29 of 36 submissions pass verification. Selecting the lowest-perplexity agent design per task yields an average 3.4% perplexity increase over BF16 and 77.33% downstream accuracy, compared with 79.56% for BF16. Recorded experiments reveal how agents achieve these outcomes and where improvement breaks down. Agents lower PPL in 21 of 24 revisions at nearly fixed storage; complete-model feedback reverses 17 of 79 prior local rankings; and generation and inference tests expose failures that likelihood alone misses. The results identify representation design, experimental judgment, and capability validation as central to agent-driven model compression, a measurable component of efficiency-oriented self-improvement.