FlowMaker improves efficiency of AI task workflows reducing costs
FlowMaker: From Tasks to Cost-Efficient Generative AI Workflow Deployments
Machine Learning
Summary
Running tasks with generative AI often involves connecting different steps, each done by a language model, to complete a bigger job. The authors found that picking the best way to build and run these steps together, while thinking about how many models are needed and how busy the computers are, helps save time and money. They created FlowMaker, a system that quickly chooses the best setup automatically. Using FlowMaker, AI workflows run faster and cheaper compared to previous techniques.
What this means in practice
- •For cloud platform operators: Optimize deployment of AI workflows by dynamically selecting models and configurations to reduce operational cost and speed up response time.
- •For enterprise software developers: Build cost-efficient AI-driven applications that automatically adapt workflow structure and model use to current resource availability and task demands.
Authors
Dimitrios Liakopoulos, Prasoon Sinha, Neeraja J. Yadwadkar
Abstract
Generative AI workflows execute user tasks by composing sequences of LLM calls, where each call performs a sub-task (e.g., summarization or translation), forming a workflow. Executing such workflows requires decisions over workflow structure, model assignments, and deployment configurations, which jointly determine latency, cost, and accuracy under dynamic workloads and resource constraints. Existing systems optimize these decisions in isolation or fix them early, leading to suboptimal cost and goodput. We make two key observations. First, efficient serving requires jointly exploring all combinations of workflows, model choices, and deployment configurations under dynamic cluster state. Second, reuse of deployed LLMs is critical for reducing cost, but must be performed on-the-fly, as reuse depends on current load and resource availability. Together, these make selecting a workflow-variant-a combination of workflow, models, and deployment configuration-a time-critical problem. We present FlowMaker, which exposes a task-level interface and selects workflow-variants on behalf of users. To realize this, we design a constraint-guided load-aware pruning algorithm for fast, on-the-fly decisions. FlowMaker improves goodput by up to 10.14$\times$ and reduces cost by up to 2.77$\times$ compared to state-of-the-art baselines, including Murakkab.