STEVE improves stability of automated prompt optimization methods
STEVE: Stabilizing Textual Gradient-Based Prompt Optimization via Error-Driven Refinement and Regularized Verification
Computation and LanguageArtificial IntelligenceMachine Learning
Summary
Creating better prompts for AI language models using automatic feedback can sometimes be unstable, causing the prompts to perform worse. The authors identify two main causes: noisy feedback from prompts that already work well and focusing too much on hard cases, which hurts performance on easier ones. They introduce STEVE, a method that fixes these problems by only updating prompts based on mistakes and carefully checking that improvements don’t cause new problems. Tests across many tasks and models show that STEVE makes prompt improvements more reliable and effective.
What this means in practice
- •For machine learning engineers: Develop more stable and effective prompt tuning techniques for improving AI model responses on diverse tasks.
- •For ai product developers: Create AI applications with more reliable and robust prompt-based interactions by reducing performance regressions during prompt updates.
Authors
Yifan Xu, Yixuan Li, Xinzhuo Li, Yixin Gu, Yifan Shen, Lijun Yu, Haohan Wang
Abstract
Textual-gradient methods automate prompt optimization through natural-language feedback, but their iterative updates can be unstable. We identify two sources of this instability: noisy gradients produced from already-correct examples and over-specialization to hard cases that degrades performance on simpler inputs. We introduce STEVE, a stabilization framework with two coupled mechanisms. Error-Driven Refinement generates gradients only from incorrectly handled examples, concentrating updates on informative failures. Regularized Verification treats every update as provisional and accepts it only when improvement on hard cases does not cause unacceptable regression on a preservation set. Across ten reasoning benchmarks, three evaluator/optimizer models, and established prompt-optimization baselines, STEVE reduces degradation and produces more robust prompts. Additional evaluations with gpt-5.4-mini/gpt-5.4 on symbolic reasoning, GSM8K-Platinum, and DS-1000 show that these gains persist with newer models and larger test sets. STEVE therefore provides a practical way to improve the stability and effectiveness of textual-gradient prompt optimization.