LLM-Driven CI-CD Workflow Intelligence for Cyber Systems Engineering
2026-07-06 • Software Engineering
Software EngineeringArtificial Intelligence
AI summaryⓘ
The authors studied how automated software build and delivery processes (CI/CD workflows) work by analyzing many projects on GitHub. They used AI models to find patterns and problems in these workflows, discovering many issues related to reliability and maintenance. They also found that the way workflows are set up varies depending on the programming language and project type. Their system can suggest improvements for these workflows, showing that understanding these processes needs more than just labeling their steps. Overall, the authors highlight the importance of combining automatic analysis with human insight to improve CI/CD workflows.
CI/CD workflowscontinuous integrationcontinuous deliverylarge language modelsworkflow analysisconfiguration filesanti-pattern detectionsoftware reliabilitysoftware maintainabilityrecommendation generation
Authors
Bonan Shen, Jiazhou Gao, Tao Ning, Wei-Jung Huang, Xin Liu
Abstract
CI/CD workflows have become executable operational policy: they decide what gets built, tested, released, and deployed, and they mediate how maintainers interact with delivery infrastructure. That makes them an important measurement point for cyber-systems engineering. Recent large language model (LLM) work shows that workflow stages can be recognized directly from configuration files, but stage labels alone do not tell us whether a workflow is brittle, unusual for its ecosystem, or worth revising first. We present an LLM-based CI/CD analysis pipeline that combines repository enrichment, anti-pattern detection, stage mining, and recommendation generation over a large GitHub corpus. Starting from 59,550 repositories with at least 1,000 stars, we identify 34,225 projects with CI/CD and collect 127,559 configuration files. Across 75,201 analyzed workflows, the anti-pattern detector reports 434,769 findings, dominated by reliability and maintainability issues. Across 59,906 configurations, stage usage differs significantly by language ($χ^2 = 4168.88$, $p < 0.001$, Cramer's $V = 0.063$), and domain analysis shows distinct operational profiles, including higher release and cache usage in mobile projects. For repository-level recommendation generation, few-shot prompting performs best overall, averaging 8.25 recommendations per repository with 96.1% YAML-valid snippets. Taken together, the results argue for CI/CD observability that combines diagnosis, context, and human review rather than treating workflow mining as a stage-classification problem alone.