Papers for

software engineering trainers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

AI Cosmos improves learning with scenario based agent interactions

AI Cosmos: An Agentic Infra for Situated Learning

Abstract: Large language models (LLMs) can improve task performance during learning, yet these gains do not necessarily reflect learners' ability to thereafter apply the knowledge and skills independently. We present AI Cosmos, an agentic infrastructure for situated learning that organizes learning around two complementary principles: scenario-based participation and cumulative assets of practice. AI Cosmos organizes learners and role-specialized agents within learning scenarios that integrate task-relevant knowledge and, when needed, executable workbenches. An asset-memory loop preserves the processes and outcomes of practice as reusable assets with explicit provenance and dependencies. We evaluate scenario-based interaction through two preregistered randomized controlled studies examining the acquisition of legal reasoning and programming competence, holding the same underlying LLM across conditions. AI Cosmos achieves greater learning gains than a conventional LLM in legal reasoning and higher mastery of unseen programming problems than an LLM-as-Teacher baseline.

Thu 29 OctComputation and Language
The gist
Learning with big language models can help people do tasks better during training, but that doesn’t always mean they can do those tasks well later on their own. The authors created AI Cosmos, a system that sets up learning like real-world scenarios with agents that have specific roles, plus tools to practice tasks directly. This system keeps track of what was learned and how, so learners can build on previous work. The authors tested AI Cosmos on legal reasoning and programming skills, finding it helped learners improve more than just using a language model as a teacher.
Open → 2610.40031v1

ChatGPT usage diversity linked to better security assignment scores

Exploring the Role of Security Experience and ChatGPT Usage Strategies on Secure Software Engineering Education

Abstract: The rapid adoption of Large Language Models (LLMs) is reshaping software engineering education, but their role in secure software engineering education remains underexplored. We report an exploratory empirical study of how 26 graduate students in a part-time MSc Cybersecurity programme used ChatGPT during a vulnerability-fixing assignment. To characterise ChatGPT use, we analysed students' ChatGPT interaction logs using a structured double-coding procedure and examined whether usage patterns and prior cybersecurity expertise were associated with assignment performance. The results show that students with varying levels of cybersecurity expertise used broadly similar ChatGPT strategies. Individual usage patterns showed descriptive differences by grade, but none remained statistically significant after correcting for multiple comparisons. In contrast, diversity of ChatGPT usage, i.e., the number of distinct usage patterns adopted, was positively associated with performance, even after controlling for cybersecurity expertise. These exploratory findings suggest that the way students engage with ChatGPT may be more informative than whether they use it, and motivate future controlled studies to guide students toward effective LLM use in secure software engineering education.

Thu 10 SeptSoftware Engineering
The gist
Using ChatGPT is becoming common in software education, but how it helps with security tasks is not fully known. The authors studied 26 cybersecurity students using ChatGPT to fix code problems. They found that it wasn’t just whether students used ChatGPT that mattered, but how many different ways they used it. Students who used a wider variety of ChatGPT strategies tended to get better grades, regardless of their prior experience. This suggests teaching students to interact with AI tools in different ways might improve learning about secure coding.
Open → 2609.11303v1