Compression reveals how much fine-tuning data a model needs
The Behavioral Rate of Fine-Tuning: Compression as a Measure of Useful Information
Machine Learning
Summary
When computer models like language AIs are fine-tuned, the amount of information needed to keep their improved behavior is unclear. The authors use a method called LoRA, which changes models in a small way, to measure how tiny a compressed update can be without losing performance. They find that the amount of useful information depends on both the data used to train and the original model, not just on simple factors like size or error. Finally, compressing these updates can even help remove noise and improve how well the model works on new tasks.
What this means in practice
- •For machine learning engineers: Quantify how much compressed fine-tuning data is truly needed to preserve model behavior across different models and datasets.
- •For ai model deployers: Use compression to reduce noise in fine-tuned adapters and improve out-of-distribution generalization of large language models.
Authors
Joey David, Paul Caillon, Alexandre Allauzen
Abstract
Low-Rank Adaptation (LoRA) adapts a frozen language model through a compact learned update, making it a natural setting for asking how much information fine-tuning needs to write. Yet adapter rank, parameter count, or numerical reconstruction error do not directly tell us how many bits are required to preserve the acquired behavior. We therefore define an operational behavioral rate $R^\star$ as the smallest serialized size of a compressed adapter that, once reloaded into the frozen base model, retains a fixed fraction of its held-out fine-tuning gain. Across LoRA fine-tuning experiments, this rate increases with distinct training content at fixed optimization budgets, but is not determined by intuitive metrics like source description length, fine-tuning gain, or weight-space reconstruction error. Most importantly, the same datasets exhibit substantially different, and even reordered, rates across frozen base models, showing that fine-tuning compressibility is a property of the dataset--receiver pair rather than of the dataset alone. However, we show that correction spectra measured on the data are able to rank these rates within a receiver. Finally, we study a striking noise-reducing effect of compression: adapters trained on deliberately corrupted datasets fall far below the frozen model, yet after compression score above it (23.7% to 82.9% on GSM8K, against 77.6% for frozen Qwen2.5-7B), consistently across tested families and sizes. The damage is unevenly concentrated in the adapter's leading singular directions, which compression appears to filter. Away from its training task, the compressed corrupted adapter also generalizes better than a clean fine-tune: on harder math it retains 97% of base accuracy, versus 44% for the clean adapter.