Framework improves prediction of github actions workflow steps
FLOWSTEP: A Framework for Learning to Predict GitHub Actions Workflow STEPs
Software Engineering
Summary
Setting up software build and deployment pipelines can be difficult because workflows include reusable parts (Actions) and project-specific code (Scripts). The authors show that treating these two parts differently helps machines better predict the next steps needed in workflows. Their approach, FLOWSTEP, uses large language models combined with contrastive learning to recognize relationships between workflow parts. This method improved prediction accuracy significantly and can also use language models to evaluate the quality of generated steps, agreeing well with human judges.
What this means in practice
- •For software development teams: Improve automation by accurately predicting the next steps in GitHub Actions workflows, reducing errors and configuration complexity.
- •For software tool vendors: Enhance CI/CD pipeline tools with smarter suggestions for workflow steps by integrating FLOWSTEP’s approach to differentiate reusable actions and custom scripts.$Commercial implications: Enables development of advanced CI/CD assistance tools that improve workflow setup for development teams, offering competitive product differentiation.
Authors
Jiangnan Huang, Bin Lin, Yutaro Kashiwa
Abstract
Continuous Integration and Delivery (CI/CD) are fundamental to modern software development, yet configuring pipelines such as GitHub Actions (GHA) workflows remains complex and error-prone. Recent work has explored Transformer-based models for GHA workflow recommendation, but existing approaches typically model workflow steps as a single homogeneous generation task. This formulation overlooks a key distinction: reusable Actions are structured, ecosystem-level components, whereas project-specific Scripts encode local build, test, and deployment semantics. Consequently, current models struggle to capture the different forms of knowledge required for accurate workflow completion. In this work, we present FLOWSTEP, a framework for GHA workflow step completion that explicitly distinguishes between Action and Script generation. FLOWSTEP combines large language models (LLMs) with contrastive learning to better capture semantic relationships among workflow contexts, Actions, and Scripts. Evaluated on established benchmarks, FLOWSTEP improves end-to-end workflow step prediction accuracy from 38.23% to 51.09%. Beyond prediction accuracy, we further study whether LLM-as-a-judge can provide a more semantic evaluation of generated workflow steps. By comparing LLM-based assessments with human annotations, we find that the best-performing model configuration reaches substantial agreement with human ground truth (Krippendorff's alpha = 0.701), suggesting that LLM-as-a-judge can serve as a scalable complement to manual evaluation.