LLM agents that adaptively choose workflows to cut costs
CB-Orchestrator: Adaptive Workflow Optimization for LLM Agents via Contextual Bandits
Machine Learning
Summary
Large language models (LLMs) can perform complicated tasks by following step-by-step procedures called workflows, but designing these workflows by hand takes a lot of work and doesn’t easily adjust to new questions. The paper presents CB-Orchestrator, a system that first creates many possible workflows and then uses a smart method called contextual bandits to pick the best one for each task. This approach reduces the time and expense needed to train these models and improves their accuracy on different benchmark tests. The method also saves a significant amount of computational resources during training compared to previous techniques.
What this means in practice
- •For software engineers: Develop AI applications that efficiently tailor workflow sequences to diverse user queries, saving compute costs during development.
- •For cloud ai service providers: Reduce API usage costs by implementing adaptive workflow selection frameworks to optimize responses for client requests.$Commercial implications: Makes cost-effective AI workflows possible for cloud providers by cutting expensive token usage while maintaining quality.
Authors
Jiahang Sun, Zhiwei Shang, Zhipiao Liu, Hongwei Yang, Guoqing Xie, Shuang Qiu, Zhongxiang Dai
Abstract
Large Language Model (LLM) agents have demonstrated remarkable capabilities in tackling complex tasks through agentic workflows. However, manually designing these workflows is labor-intensive and lacks flexibility to adapt to diverse queries. While automated workflow optimization is a promising direction, existing methods often incur expensive API costs during training-phase feedback collection. Furthermore, they fail to adaptively reuse high-quality workflows, which creates a bottleneck for both performance and cost-effectiveness. To address these limitations, we propose CB-Orchestrator, an adaptive framework that decouples workflow generation from selection. In the first stage, we construct a diverse pool of candidate workflows via evolutionary search. In the second stage, we formulate workflow selection as a Contextual Bandit problem, which enables sample-efficient learning by balancing exploration and exploitation, thereby significantly reducing the feedback required for training. During inference, the model adaptively selects the optimal workflow tailored to each query. Evaluations across five diverse benchmarks demonstrate that CB-Orchestrator consistently outperforms all baselines. Notably, on the MATH dataset, CB-Orchestrator achieves better performance than strong recent baselines while reducing training-phase API token overhead by no less than 40.55% and total end-to-end training (including workflow pool generation) token overhead by no less than 29.73%.