Model blending improved with safe rotary embedding adjustment dial
MeRoTune: RoPE-Safe Merging with a Tunable Dial
Computation and Language
Summary
Sometimes, when two versions of a language model are combined by averaging their settings, parts inside don't match perfectly, especially around the way positions are handled. The authors found that a common fix for this mismatch only works if a certain correction fits very specific rules related to how position information is rotated. They figured out which kinds of adjustments work exactly and made a new way for two fine-tuned models to each learn their own safe corrections. This lets users smoothly blend models at different mix levels after training, like turning a dial, instead of being stuck with one fixed mix.
What this means in practice
- •For machine learning engineers: Combine different fine-tuned language models safely while retaining positional embedding integrity using a tunable blend control.
- •For ai service operators: Provide users the ability to adjust model blend levels dynamically after deployment, improving flexibility without retraining.
Authors
Salman Faroz
Abstract
When you merge two fine-tuned models from the same base checkpoint by simply averaging their weights, you implicitly assume their attention subspaces are still aligned. Recent work attempts to fix misalignments by learning an invertible correction matrix, $M$, for each model's query and key projections. This correction cancels out---using $M$ on the query side and $M^{-T}$ on the key side---right before the dot product. However, this cancellation is only exact if nothing sits between the projection and the dot product. In reality, almost all modern open-weight language models put a rotary position embedding (RoPE) exactly there. In this paper, we show that this cancellation is exact under RoPE if and only if $M$ commutes with RoPE's per-position rotation. We derive the specific class of matrices where this holds: a scaled rotation acting independently within each RoPE frequency pair. This forms a strict, low-dimensional subset of the unconstrained matrices that current methods normally train. Building on this, we turn this constrained matrix class into a new merging method. While keeping the base weights entirely frozen, two fine-tunes each learn their own RoPE-compliant correction matrices. We optimize these corrections against a chosen blend ratio so the final result can be adjusted post-hoc like a dial, rather than locked into a single fixed merge. Our default approach trains at one fixed blend ratio, similar to how LoRA sets its scaling hyperparameter in advance. We also experiment with resampling the blend ratio randomly at every training step, and we report the results of both approaches.