Papers for

software analytics vendors

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Chatbot using GPT-4 helps everyone access software repository data

A Tool-Augmented, GPT-4 Chatbot for Real-Time Repository Data Analysis

Abstract: Software repositories contain vast amounts of data on code contributions, bug reports, and project activities, yet this information remains challenging for non-technical stakeholders and developers to access due to limited expertise in querying repositories. To address this, we introduce a novel chatbot architecture leveraging OpenAI's GPT-4 model for automated extraction and analysis of repository data. In contrast, our architecture takes a structured path first by parsing the user's query to extract relevant parameters, then selecting the correct tool to employ based on that analysis, and finally invoking the GPT-4 model to create a highly detailed response. In contrast to previous work based on multi-component systems with embedding models and document retrievers, our architecture inverts the process by relying on prompt engineering and tool selection to fit with the query intent. To validate our approach, we conducted experiments on various question types, including Issues, Pull Requests, Commits, Compound Questions, and General Repository Information, evaluating our target prompts' ability to improve the accuracy of responses from the model. Beyond demonstrating the utility of this architecture to a diverse set of users, our findings suggest that this architecture can make repository data more accessible to technical and non-technical audiences through the production of actionable insights.

Mon 7 SeptArtificial IntelligenceSoftware Engineering
The gist
Software project data like bug reports and code changes are hard for many people to explore without technical know-how. The authors designed a chatbot that breaks down user questions to pick the right tools before using GPT-4 to give detailed answers. This approach helps people with different backgrounds get useful insights from complex repository data. Their tests showed it works well with various question types and can make software project information easier to understand.
Open → 2609.07586v1