Avatar improves scientific workflow efficiency with AI orchestration
Avatar: Toward Autonomous End-to-End Orchestration of Scientific Workflows using LLMs
Distributed, Parallel, and Cluster ComputingMultiagent Systems
Summary
Scientific workflows automate research tasks but usually follow fixed rules. The paper presents Avatar, a system that uses AI (large language models) to decide how to run and manage these workflows more flexibly. Avatar can switch between traditional rule-based control and AI-driven decisions, reducing wasted computation and GPU busy time significantly. This work shows a way for scientific software to manage itself more intelligently instead of just following preset instructions.
What this means in practice
- •For workflow engineers: Integrate Avatar to automate and optimize execution control in scientific workflow systems, reducing resource waste and GPU idle time.
- •For cloud infrastructure teams: Use Avatar’s flexible orchestration to improve scheduling and resource efficiency of compute-heavy scientific tasks on shared hardware.
Authors
Suman Raj, Hai Duc Nguyen, Haochen Pan, Ryan Chard, Kyle Chard, Ian Foster
Abstract
Scientific workflow management (WMSs) systems automate execution, yet orchestrate using fixed, hand-tuned rules. LLM agents promise more autonomous orchestration, but it remains unclear where to introduce agentic reasoning, how to bound its risk, and when it actually helps. We present Avatar, an actor-based architecture comprising an orchestrator, an executor, and a provenance monitor. Each actor's decision policy is pluggable (rule-based or LLM-backed) via a single adapter-validated action catalog, so conventional and agentic control run on the same core across different WMSs. We present an implementation using the Academy framework and evaluate Avatar across three workloads. We observe that Avatar's rule mode reproduces native execution, with a single unchanged core running all three. Moreover, LLM-backed Avatar reports a reduction of compute wastage by $55\%$ and cuts GPU-busy time by $40\%$. Overall, we envision Avatar as a step toward workflow systems that reason about their own orchestration rather than follow pre-fixed rules.