NormTune improves efficient tuning for hard reasoning tasks
NormTune: Norm-Guided Parameter-Efficient Fine-Tuning for Reasoning-Intensive Tasks
Artificial Intelligence
Summary
Fine-tuning AI models to solve complex reasoning problems usually needs a lot of computer power. The authors found that cheaper methods to adjust these models often update their internal settings too little, leading to worse results. They introduced NormTune, a way to boost these smaller updates selectively, improving performance. Tests show that NormTune helps different tuning techniques get much closer to the full, expensive tuning results.
What this means in practice
- •For machine learning engineers: Enhance parameter-efficient tuning processes to improve model accuracy on complex reasoning problems without heavy GPU use.
- •For ai service providers: Deploy more cost-effective model updates that approach full fine-tuning quality for reasoning tasks in production AI systems.$Commercial implications: Provides a method to improve fine-tuned AI services' reasoning capabilities with reduced computational costs, enabling better AI products.
Authors
Zhangming Li, Qinghao Hu, Weifan Guan, Yifan Zhang, Peisong Wang, Jian Cheng
Abstract
Full fine-tuning has achieved remarkable success on reasoning-intensive tasks, but it often requires substantial GPU resource consumption. Parameter-efficient fine-tuning (PEFT) provides a promising alternative for reducing this cost, yet we observe a substantial performance gap on reasoning-intensive tasks. In this paper, we present three key observations showing that this gap is closely associated with norm-related under-adaptation, where PEFT produces markedly smaller weight-update norms than full fine-tuning. Based on these observations, we propose Norm-Guided Parameter-Efficient Fine-Tuning (NormTune), a simple yet effective strategy that adaptively strengthens insufficiently updated layers according to their weight-update norms. Extensive experiments show that NormTune consistently enhances multiple PEFT methods across challenging reasoning benchmarks. Specifically, when trained on S1K-1.1, integrating NormTune into rank-8 LoRA, Bone, and MoRA yields substantial average accuracy improvements of 24.24\%, 11.33\%, and 16.21\% , respectively, across AIME 2024, MATH 500, GPQA, and AIME 2025. Moreover, NormTune-enhanced rank-256 LoRA, Bone, and MoRA achieve performance comparable to S1 full fine-tuning, with respective average gains of 2.17\%, 0.93\%, and 3.00\% over this baseline.