Software change request classification faces data and method challenges

Software Change Request in AI-Orchestrated Software Evolution: Literature Review and Research Agenda

Software Engineering

Summary

Mobile app stores get millions of user suggestions and problems, called Software Change Requests (SCRs), that help improve apps. The study reviewed how machines learn to identify and sort these requests from user reviews. The researchers found that common methods like Random Forest and Support Vector Machines are mostly used, while newer deep learning methods are less common. They also noticed that available datasets vary a lot and there is no agreement on how to classify these requests, which makes automation hard. The authors suggest clearer categories, shared data, and better labeling to improve how apps evolve.

What this means in practice

A survey. It maps existing work.

Authors

Muhammad Nadeem, Khurram Shahzad, Muhammad Nadeem Majeed

Abstract

In the emerging paradigm of holistic AI-orchestrated software maintenance and evolution, large-scale app stores continuously generate exponentially vast Software Change Requests (SCRs) streams through user feedbacks. These streams enable intelligent software evolution. Automating SCR extraction and classification is crucial with rapid growth of Mobile Apps (MA). However, significant challenges exist such as scare large-scale datasets, inconsistent classification schemes, limited data sharing and insufficient research. This study presents a first and thorough systematic survey of SCR identification from MA Reviews (MAR). In particular, we investigated how Machine Learning (ML) and Deep Learning (DL) enhanced identification and classification of SCR. This study has found pertinent datasets varying sizes from 5,000 to more than 18 million reviews. The key findings highlight the dominant use of Random Forest and Support Vector Machines, limited adoption of deep learning techniques, inconsistent F1-score results ranging from 0.08 to 1 and partial automation in existing SCR tools such as SURF and ChangeAdvisor. Finally, we have performed a synthesis of the studies to presents a comparative taxonomy of SCR methodologies, identify research gaps and annotation frameworks. We contend that classification schemes, shared datasets and annotation frameworks are essential for advancing SCRs and thereby improving the maintenance of mobile applications.