GEPA merge helps when finely tuned to models and tasks
When Does GEPA Merge Help? A Mechanism-Level Diagnostic
Computation and Language
Summary
GEPA is a tool used to improve prompts for AI language models, but its merging step sometimes helps and sometimes hurts depending on the AI model. The authors looked at why this happens and found that it depends on how the merging is set up, not a broken method. They also tested if features of the prompts could predict when merging would work well, but they couldn’t. Finally, they made a way to spot risky merges and avoid big mistakes, though it doesn’t always beat just skipping merging.
What this means in practice
- •For language model developers: Adjust GEPA merge settings to improve prompt optimization across different AI models without causing performance drops.
- •For ai system integrators: Use behavior-based probes to detect risky prompt merges and reduce failures in multi-model AI deployments.
Authors
Chong Zhao, Ruyi Yang, Changming Liu, Alice Zhao
Abstract
GEPA is among the strongest reflective prompt optimizers for language-model programs, but its system-aware merge operator is unstable across models: merge improves gpt-4.1-mini on average while degrading Qwen3-8B, an asymmetry noted but left unexplained by the GEPA paper. We investigate this instability across five datasets and two models. Our nine-configuration sweep over merge timing and merge operator shows that merge can improve both models when configured appropriately. This suggests that merge instability is better explained by configuration sensitivity than by an inherent flaw in the merge operator. We then test whether prompt-level features predict GEPA's merge-accept decisions and find that they do not, ruling out static-feature gating. As a proof of concept, a behavior-based probe built from these diagnostics avoids large merge failures in our multi-seed runs by abstaining on high-risk tasks, although it does not consistently outperform NoMerge. Code is available as GEPA-MergeProbe at https://github.com/ChongZZZZ/GEPA_MERGE .