Adaptive feedback improves prompt tuning for language tasks
AdaGEPA: Adaptive Feedback Allocation for Reflective Prompt Optimization
Computation and Language
Summary
Language models can work better if you give them smarter instructions called prompts. The authors found that when picking examples to improve these prompts, choosing examples based on where the prompt is weak helps a lot. Their new method, AdaGEPA, adjusts which examples to focus on during updates so that the prompt gets better overall, not just on a few cases. Tests showed AdaGEPA made prompts work better and faster on several tasks, including task dialogues.
What this means in practice
- •For language model developers: Improve prompt tuning processes by adaptively selecting feedback examples to efficiently enhance language model task performance.
- •For customer support ai teams: Boost dialogue system accuracy across services by using adaptive feedback to optimize prompts for better goal completion.
Authors
Junyang Chen, Zecheng Wang, Jingbang Chen
Abstract
Prompt optimization improves the performance of language-model systems on downstream tasks by refining their prompts. Classical methods evaluate prompts on task examples and use the resulting feedback to guide prompt revisions through reflection. However, when feedback selection does not account for the prompt's weaknesses, these revisions may improve performance on selected examples without yielding broader task improvements. To address this issue, we propose AdaGEPA, an adaptive feedback-allocation method that uses the prompt's performance and task structure to select examples for the next prompt revision. Our method replaces at most one example in each feedback minibatch to target an identified weakness while preserving the remaining feedback context. Across our main experiments on six downstream benchmarks, AdaGEPA achieves higher mean validation scores than non-adaptive feedback selection under matched rollout budgets. AdaGEPA also finds high-performing prompts earlier across several tasks. In the initial Schema-Guided Dialogue (SGD) study, its half-budget prompts outperform the non-adaptive baseline's full-budget prompts in joint goal accuracy on new dialogues from services seen and unseen during search. Overall, our findings highlight the potential of adaptive feedback allocation to improve both the effectiveness and rollout-budget efficiency of reflective prompt optimization.