Large language models show promise and limits in automated code refactoring
Software Refactoring Research with Large Language Models: A Systematic Literature Review
Software Engineering
Summary
Code refactoring means improving the internal structure of software without changing what it does. Developers often skip this because it takes a lot of effort. The authors reviewed studies that use large language models (LLMs) to automate refactoring and found that these models can reduce common coding problems like bad design up to 89% in some cases. However, the models sometimes make mistakes, struggle with complex changes, and misunderstand instructions. The research also highlights the need for better ways to measure success in refactoring efforts.
What this means in practice
- •For software developers: Use LLM-based tools to automatically perform common refactoring tasks like Extract Method to reduce code complexity and improve code quality.
- •For software testing teams: Employ datasets and code smell detection tools analyzed in LLM refactoring research to evaluate and benchmark code maintainability.
A survey. It maps existing work.
Authors
Sofia Martinez, Luo Xu, Mariam Elnaggar, Eman Abdullah AlOmar
Abstract
Background: Code refactoring is the improvement of code internally without changing the external functionalities of the program. Due to its exhaustive nature, developers often avoid manually refactoring code. Researchers have since looked into utilizing LLMs to automate the task of refactoring. Aim and Method: Despite the promising results, there is a lack of clear un- derstanding of LLMs' effectiveness in automated refactoring. Result: From our literature review we found that: (i) there are various tools that different studies use to enhance and study LLM-driving refactoring, with tools that were used to detect code smells, generate code bases, and compare refactoring outcomes. (ii) Various datasets were collected from multiple open- source projects in multiple programming languages for analysis. These platforms included GitHub, Apache, and F-Droid, with the most popular language collec- ted and analyzed being Java. (iii) One-Shot, Few-Shot, Context-Specific, and Chain-of-Thought prompting methods have been shown to be the most effective depending on the language used. In some instances, being capable of reducing code smell by up to 89%. (iv) The definition of "accuracy" varies significantly across the literature surveyed, as this depends on the context of the study, thus calling for a need to have a standardized measurement for Accuracy. (v) The most often mentioned code smells were Large Class and Long Method, with a lot of studies also not specifying, while the most often applied refactoring type is Extract Method, showing promising results in using LLM to perform this refactoring type. (vi) When working with LLMs, they often generate erroneous code, struggle with more complex refactoring, and often misunderstand the developers' requests and miss refactoring requests.