Automated analysis reveals common gaps in software architecture records
A Text Mining and Classification Approach for Analyzing Architecture Decision Records
Software Engineering
Summary
In software projects, teams write down important design decisions in notes called architectural decision records (ADRs). The authors developed a computer method that reads many of these records to see what kinds of decisions get written down and how well the notes follow a common template. They found that some types of decisions, like technology choices, are often recorded, while others, like alternative options and reasons, are missing or poorly documented. This method can help software architects understand and improve how they document design decisions.
What this means in practice
- •For software architects: Identify common documentation gaps in architectural decision records to improve knowledge sharing and decision quality.
- •For software project managers: Assess adherence to architectural documentation templates to ensure consistent recording of design decisions across projects.
Authors
Nicolás Miccio Palermo, Antonela Tommasel, J. Andrés Diaz-Pace
Abstract
Architectural decision records (ADRs) have become a popular lightweight mechanism for documenting architectural knowledge in software projects. However, there is limited empirical evidence on the kinds of architectural concerns captured in ADRs and how well their contents align with established architectural knowledge concepts and documentation practices. In this paper, we propose an automated text-mining and classification approach for analyzing ADRs at scale. We apply this approach to a dataset of ADRs extracted from ~550 open-source repositories, combining topic modeling, LLM-based classification, and template compliance checks. Our analysis examines decision taxonomies and quality attributes, and the degree to which ADRs adhere to the MADR template. Our findings show that ADRs frequently capture existence, technology, and process-related decisions, while alternatives, decisions drivers, and some quality concerns remain under-documented. We also observe recurring mismatches between ADR contents and template sections. These insights into current documentation practices provide architects with valuable information to reflect on how ADRs are and should be used to effectively deal with architectural knowledge. Furthermore, our automated approach is adaptable to other architectural tasks.