Software architecture records often miss key decision details
A Text Mining and Classification Approach for Analyzing Architecture Decision Records
Software Engineering
Summary
Architectural decision records (ADRs) are notes software teams write to explain important design choices. But it's unclear what kinds of architectural ideas these notes usually include and whether they follow common writing templates. The authors created a computer tool to automatically analyze many ADRs from open-source projects. They found that while ADRs often cover what decisions were made and the technology used, they often leave out why alternatives were rejected and some quality concerns. This work helps software designers understand how people document architecture and improve their record-keeping.
Architectural decision recordsText miningClassificationTopic modelingLarge language modelsSoftware architectureQuality attributesDocumentation templatesOpen-source software
Authors
Nicolás Miccio Palermo, Antonela Tommasel, J. Andrés Diaz-Pace
Abstract
Architectural decision records (ADRs) have become a popular lightweight mechanism for documenting architectural knowledge in software projects. However, there is limited empirical evidence on the kinds of architectural concerns captured in ADRs and how well their contents align with established architectural knowledge concepts and documentation practices. In this paper, we propose an automated text-mining and classification approach for analyzing ADRs at scale. We apply this approach to a dataset of ADRs extracted from ~550 open-source repositories, combining topic modeling, LLM-based classification, and template compliance checks. Our analysis examines decision taxonomies and quality attributes, and the degree to which ADRs adhere to the MADR template. Our findings show that ADRs frequently capture existence, technology, and process-related decisions, while alternatives, decisions drivers, and some quality concerns remain under-documented. We also observe recurring mismatches between ADR contents and template sections. These insights into current documentation practices provide architects with valuable information to reflect on how ADRs are and should be used to effectively deal with architectural knowledge. Furthermore, our automated approach is adaptable to other architectural tasks.