Pid gates improve prompt updates under noisy feedback
PID-Gated Continual Prompt Adaptation Under Noisy Feedback
Machine LearningArtificial IntelligenceComputation and Language
Summary
Large language models (LLMs) need to update their instructions as tasks change, but relying on every single correction can cause mistakes to stick, especially if those corrections are noisy or wrong. The authors propose using control theory ideas from PID controllers that help decide when to update instructions based on error, accumulated evidence, and ability to undo bad changes. Their approach, called PID-gated prompt adaptation, balances making accurate updates while limiting costly edits, performing better than simpler or more complex alternatives in varied noisy scenarios.
What this means in practice
- •For llm engineers: Design prompt update systems that decide when to integrate user feedback to maintain accuracy without excessive editing costs.
- •For customer support automation teams: Improve automated response systems to adapt instructions reliably despite noisy or inconsistent user feedback.
Authors
Zhentao Xu
Abstract
Deploying LLM agents in non-stationary environments requires continually revising their instructions as the task drifts, yet rewriting after every reported mistake can turn noisy feedback into persistent errors. This raises an often-overlooked scheduling question in prompt adaptation: when should feedback trigger a lasting instruction update, and at what editor-call cost? Taking inspiration from proportional-integral-derivative (PID) control, we introduce a family of lightweight gates that react to the current error, accumulate evidence over time, and roll back deterioration after an edit. We benchmark these gates against both simple baselines and advanced prompt optimizers (TextGrad and GEPA), over a diverse suite of drifting tasks, LLM parameter sizes, feedback-noise levels, and random seeds. PID-gated continual prompt adaptation yields a more favorable Pareto trade-off between task performance (high accuracy, low error) and editor-call budget than the baselines considered, pointing to control-theoretic scheduling as a simple, general tool for robust prompt adaptation under unreliable feedback.