Model merging raises hallucination risk but MergeHEIR cuts it down
MergeHEIR: Mitigating Multimodal Hallucinations as the Tax of Model Merging
Artificial Intelligence
Summary
Combining different expert AI models into one can make the new model more likely to make up false information, called hallucination. The researchers found that all merged models they tested had higher hallucination rates than the original experts on their own. They created MergeHEIR, a technique that adjusts the merged model carefully to reduce hallucination while keeping the original skills. MergeHEIR uses small task-specific examples to guide its changes so it doesn’t lose what each expert knew. Tests show it balances reducing false information against keeping expertise better than other methods.
What this means in practice
- •For large language model engineers: Improve merged multimodal AI models by reducing hallucinations while preserving specialized task skills using minimal extra data.
- •For ai system integrators: Deploy consolidated models with better reliability in handling multiple tasks by using MergeHEIR to balance accuracy and expertise retention.
Authors
Jinyu Li, Hao Fang, Zhiming Zhang, Jiawei Kong, Bin Chen, Shu-Tao Xia
Abstract
Model merging consolidates task-specialized experts into a single deployable model. However, we show that such capability consolidation incurs a merging tax of increased hallucination: across 8 model-merging methods, every merged checkpoint exhibits a higher hallucination rate than the average of its constituent experts. An intuitive approach is to adapt existing hallucination-mitigation methods to the post-merge model, yet this unconstrained adaptation disrupts inherited capabilities, creating a tension between hallucination mitigation and expertise retention. To tackle this challenge, we introduce MergeHEIR, a post-merge adaptation framework designed to reduce this merging tax while preserving expertise inherited from initial experts. Using small expert-task calibration sets, MergeHEIR constructs layer-wise null-space projectors via SVD from task-specific activations collected from the merged checkpoint, and periodically projects the accumulated post-merge displacement onto the resulting null spaces to preserve inherited expertise. Theoretically, we establish minimum-distortion and maximum-dimensionality guarantees, characterize the threshold-controlled adaptation-retention trade-off, and extend perturbation guarantees beyond finite calibration data. Across 24 paired comparisons spanning three MLLM configurations and 8 model-merging methods, MergeHEIR consistently mitigates hallucination while largely preserving inherited expertise, demonstrating a more favorable hallucination-retention trade-off.