Decentralized Instruction Tuning: Conflict-Aware Splitting and Weight Merging
2026-06-01 • Machine Learning
Machine Learning
AI summaryⓘ
The authors study how to better train large language models on many different tasks without slowing down or getting mixed-up updates. They propose a method called MERIT that breaks the training tasks into groups based on conflicting gradients, trains each group separately without sharing data during training, and then merges the results smartly. Their theory explains why merging can reduce conflicts and improve performance. Experiments show that MERIT improves model accuracy and scales well to large, diverse datasets.
instruction tuninglarge language modelsgradient interferenceparameter mergingPCAcurvaturedecentralized trainingVision-FLANtoken-weighted averagingspectral filtering
Authors
Minsik Choi, Geewook Kim
Abstract
Instruction tuning aligns large language models, including multimodal ones, with diverse user intents, but scaling to heterogeneous mixtures is hindered by gradient interference and bandwidth-heavy synchronization. We ask whether these two bottlenecks can be addressed jointly by training parts of the mixture independently and reconciling them once in parameter space. We develop a local quadratic theory inside a shared flat basin that yields three results: weight merging produces a curvature-weighted variance reduction; PCA-aligned conflict splitting maximizes this gain along high-curvature directions; and merging additionally acts as spectral filtering with implicit norm regularization. These results directly motivate MERIT, a decentralized merge-ready instruction-tuning pipeline that estimates dataset-level gradient conflicts, partitions the mixture along the top PCA conflict axes, fine-tunes each partition independently with no inter-partition communication, and merges once via token-weighted averaging. On Qwen2.5-VL-3B with 136 Vision-FLAN tasks, MERIT improves the 8-benchmark average from 54.3 (joint training) to 57.0. The same recipe scales to a 7B model on a 1.6M-example, 176-source mixture -- matching or exceeding centralized joint training with minimal cost overhead -- and transfers to text-only FLAN. Our code is available at https://github.com/naver-ai/merit.