Calibration gains transferred between model forms preserve most accuracy improvements
Reusing Calibration Gains across Model Representations
Machine LearningDatabases
Summary
Different forms or versions of a pretrained AI model may need tuning to work well. This paper shows that when you improve one form of a model by adjusting certain settings, you can often reuse many of those improvements in another form without needing extra training. The researchers tested this on specific AI language model versions and found that most of the benefits stayed intact when transferred, saving work and effort. They also showed that keeping the improvements organized properly matters for this to work well.
What this means in practice
- •For machine learning engineers: Reuse calibration adjustments across model quantizations to reduce retraining effort for language models.
- •For ai deployment teams: Apply shared calibration gains to different model versions to efficiently maintain accuracy in production.
Authors
Xu Cao, Wenxuan Xiao
Abstract
Scale calibration learns an update for one model representation. We study how much of that update can be reused by another representation of the same pretrained model. Our transfer procedure extracts positive group-wise gains and installs them in the recipient's weights or native quantization scales without recipient-side optimization. On a quantized recipient, it preserves integer codes, zero points and storage layout. For a Qwen3-1.7B floating-point/GPTQ INT3 pair, bidirectional transfer retains 76.9-90.5% of the text-loss improvement from recipient-specific calibration on fresh WikiText and C4 cohorts excluded from project development. Shuffling the same gain values within each projection worsens both recipients, showing that their learned group assignment matters after transfer. A Llama-3.2-3B control keeps the initial native tensors at all four calibrated projections identical and changes nine projections elsewhere. Cross-installation retains 98.6-99.3% of the text-calibration benefit, alongside a small, resolved preference for each recipient's own gain. These results quantify both the adaptation carried by a reusable calibration update and the remaining benefit of recipient-specific training.