Judge-guided evolution improves agent program adaptation at test time
JET: Judge-Guided Evolution at Test Time for Agent Programs
Software EngineeringArtificial Intelligence
Summary
When computer programs control agents that interact with tools and environments, it can help if these programs adapt while they are running. The challenge is knowing which program changes are actually better when you don't have direct feedback from a final score. The authors introduce a method called JET, which first learns a judge program that evaluates behavior on known tasks, then uses this judge to guide improvements on new tasks without seeing the true scores. This approach led to better performance in tests, especially when starting from scratch, by effectively transferring the judge to new environments to help evolve the agent's program.
What this means in practice
- •For autonomous system developers: Improve adaptive control software by evolving programs guided by transferable judges without needing direct task feedback during deployment.
- •For robotics engineers: Enhance robot tool-use strategies on new tasks by evolving control programs using judges trained on prior data to guide improvements without new reward signals.
Authors
Yao Long Teng, Jiayi Cai, Bo An
Abstract
An agent's executable program governs how it uses tools, processes observations, and responds to failures. Evolving this program at test time can help adaptation, but deciding which changes to retain is difficult when true rewards are unavailable. Execution traces provide evidence of agent behavior, yet interpreting that evidence requires a judge that remains useful as tasks and candidate programs change. We introduce Judge-Guided Evolution at Test Time (JET), which evolves an executable judge on labeled source trajectories, then freezes and transfers it to guide target-side program evolution. The judge supplies scores and diagnostic feedback without target evaluator access or model-weight updates. On unseen WebShop tasks, JET achieves approximately 13% higher mean reward than fixed-rubric guidance when evolution begins from an unevolved program (cold start) and 4% higher when it begins from one already optimized on source tasks (warm start), with a 36% relative improvement in cold-start exact success. An exact-judge control on PushT, where the judge reconstructs the scoring rule from observations, shows that without judge error, program search becomes the bottleneck. Analyses identify useful reward-prediction logic in the evolved code and show that better final selection alone cannot explain the gains. These results support executable judge transfer for program adaptation under evaluator-preserving task shifts.