LLM guided evolution boosts accuracy across production AI systems
Meta-Reflective Evolution Effectively Improves Production AI Systems
Machine LearningArtificial IntelligenceComputation and LanguageSoftware Engineering
Summary
Improving AI systems by changing their programs or instructions can be tough when problems happen for hidden, complex reasons. The authors introduce CoCoEvolve, a new method that remembers past changes and checks how live AI systems work before suggesting improvements. This approach helps fix AI problems better than older ways, leading to much higher accuracy on multiple real-world AI tasks. Their tests show CoCoEvolve outperforms existing methods on data agents, pipelines, and engineering workflows, helping AIs work more reliably and precisely.
What this means in practice
- •For data engineering teams: Improve the reliability and accuracy of AI-powered data workflows by using meta-reflective evolutionary optimization to identify and fix complex workflow issues.
- •For ai platform operators: Optimize live AI systems such as data agents and pipelines by selectively using historical and live system information to guide program improvements for better performance.
Authors
Seung Won Wilson Yoo, Chang Zeng, Aditi Somayajula, Sahithi Ankireddy, Rohan Kosalge, Avishree Khare, John Mitchell, Aditya G. Parameswaran, Rajhans Samdani, Anupam Datta
Abstract
Large Language Model (LLM)-guided evolutionary optimization has successfully improved prompts, programs, and even entire AI systems. Most optimization methods select a candidate from the search frontier, then mutate it in one LLM pass based only on its failures. This focus on the frontier can stall on production AI systems, where isolated failures reveal little about complex, hidden interactions. We introduce CoCoEvolve (Cortex Code Evolve), an evolutionary optimizer built around what we call meta-reflection, wherein an agentic mutator (Cortex Code) selectively retrieves evidence from the entire evolutionary history and probes the live system before verifying and proposing mutations. We apply CoCoEvolve to optimize three classes of production AI systems: data agents, LLM-powered data pipelines, and data engineering workflows. On DABStep, CoCoEvolve tops the public leaderboard, reaching 92.8% held-out hard accuracy compared to 78.4% for Meta-Harness and 30.4% for GEPA under matched conditions. Across six LLM data pipeline workloads, it outperforms every evaluated baseline. On data-eng-bench, CoCoEvolve lifts test-set solve rate from 59.1% to 77.4%, past the best baseline of 73.6%. It also has 11 percentage points higher accuracy than GEPA on HotpotQA while remaining competitive with specialized optimizers on established mathematical and systems benchmarks. Matched ablations confirm that each component of CoCoEvolve contributes to these gains, demonstrating the value of meta-reflection for optimizing production AI systems.