How Software Engineering Students Use LLMs to Write Research Papers: An Experience Report
2026-06-03 • Software Engineering
Software Engineering
AI summaryⓘ
The authors describe how they included large language models (LLMs), like AI text helpers, in a software engineering class assignment. Students had to write a short research paper using specific review methods and explain how they used LLMs throughout their work. The authors studied 146 student reports and found that students used LLMs for brainstorming, clarifying methods, organizing ideas, and improving writing, but they also worried about mistakes in the AI-generated content. The paper shares lessons about teaching software engineering with AI tools.
large language modelsempirical software engineeringsoftware architecturerapid reviewgray literature reviewAI-assisted learningreflective practiceeducational technologydisclosure statements
Authors
Ronnie de Souza Santos, Maria Teresa Baldassarre, Cleyton Magalhaes, Italo Santos
Abstract
Large language models are increasingly becoming part of software engineering education, including activities involving empirical software engineering and evidence synthesis. This paper reports an educational experience involving the integration of reflective LLM use into an empirical methods assignment in a third-year software architecture course. Students were asked to develop a short research paper using either a rapid review or a gray literature review methodology and to disclose how LLMs were used throughout the assignment. We analyzed 146 student disclosure statements using a cross-analysis process combining LLM-assisted categorization with manual verification and refinement by the researchers. The reflections describe how students incorporated LLMs during activities such as brainstorming, methodological clarification, organization of findings, and writing refinement, while also reporting concerns regarding inaccuracies and verification of generated content. This experience report discusses lessons learned and educational implications for integrating AI-assisted technologies into empirical software engineering education.