Recovering Software Architecture Intent from Historical Work Items using Generative AI: A Mixed-Methods Industry Case Study
Software Engineering
Summary
The authors explore how to create architecture diagrams from written work items like user stories in agile software projects. They use a step-by-step method powered by large language models (LLMs) to turn unstructured text from tools like Azure DevOps into C4 architecture diagrams. Their approach was tested on two real projects, and experts found the diagrams helpful for understanding system design and spotting differences between planned and actual code. The results show good accuracy in identifying system components but less consistency in showing relationships between them. Overall, the authors demonstrate that LLMs can assist in recovering software architecture from project documentation.
Software ArchitectureC4 ModelLarge Language ModelsAgile MethodologyWork ItemsAzure DevOpsArchitectural RecoveryTraceabilityChain-of-Thought ReasoningArchitectural Drift
Authors
Dominik Storck, Tobias Eisenreich, Stefan Wagner
Abstract
Software architecture is often only partially captured in code, while much of the design intent lives in evolving project artifacts. In agile projects, work items, user stories, and related tracking documents preserve valuable traces of that intent, but they rarely support direct architectural analysis. This work investigates the recovery of C4 architecture diagrams from historical agile work items using an LLM-based pipeline. The semi-automatic five-step workflow employs a prompt chain, bidirectional traceability, and Chain-of-Thought reasoning to transform unstructured Azure DevOps work items into visual artifacts. Evaluated on two industry projects, we use a mixed-methods design combining qualitative expert interviews with a quantitative stability analysis. Practitioners perceive the generated architectural baselines as accurate and highly useful for system comprehension. Strictly bound by their input data, the artifacts mirror the documented intent, thereby surfacing discrepancies and architectural drift when compared to the implemented reality. Quantitatively, the workflow exhibits high stability for architectural entities but lower stability for their relationships, with relative variance compounding across generation steps. The proposed workflow demonstrates the practical viability of LLM-assisted architectural recovery based on development process artifacts.