LLM-based system improves software requirement change impact analysis accuracy
DRILS: Dependency-based Requirement Change Impact Analysis Using LLMs
Software Engineering
Summary
Software requirements often change, and knowing which parts of the system are affected by those changes is hard and takes a lot of time. The authors present DRILS, a system that uses advanced language AI models working together to find and judge which requirements will be impacted by a change. It also uses a graph of requirement dependencies and focused context to improve results. Their tests show DRILS finds more correct impacts and makes fewer mistakes than existing methods, helping teams save effort in managing changes.
What this means in practice
- •For software engineering teams: Identify affected software requirements after changes to speed up impact assessment and reduce manual review.
- •For regulatory compliance teams: Track and assess requirement changes related to regulations more accurately to ensure compliance with less manual effort.
Authors
Romina Etezadi, Ikram Darif, Lionel Briand, Ramesh S, Parkish Peranandam
Abstract
Change is inevitable in software systems, where requirements must continuously adapt to evolving stakeholder needs, regulations, and project scopes. Managing such changes is a fundamental software engineering challenge, as manually identifying the impact of a change request, given high requirements volume, Natural Language (NL) complexity, and intricate interdependencies, is error-prone and time-consuming. Large Language Models (LLMs) offer promising solutions for requirements change impact analysis (CIA), yet their effectiveness when enhanced with advanced techniques remains underexplored. We present DRILS, an LLM-based agentic framework for requirements Change Impact Analysis, employing a multi-agent architecture to generate candidate impact sets, with an LLM-as-a-judge promoting diversity during selection. DRILS features two key components: RE-RAG, a carefully engineered RAG technique providing focused domain context, and a requirement dependency graph (RDG) as an additional knowledge source. Evaluation across three manually curated datasets demonstrates superior performance over state-of-the-art baselines, achieving an average recall of 87.1%, precision of 76.3%, and F1 score of 81% at a lower cost. Precision gains are particularly significant, 41.1%, 83.4%, and 149.8% over the closest baseline per dataset, as higher precision directly reduces false positives and manual validation effort. We also contribute three annotated CIA datasets to support future research.