ChatGPT boosts programming scores but hinders memory and ownership
Your Programming Students' Cognition with ChatGPT: Higher Performance, Lower Retention, and Reduced Ownership
Computers and Society
Summary
Using ChatGPT helps students write better code on programming tasks, but it makes them remember less of what they learned afterward. The researchers found that students who used ChatGPT scored higher on their coding tasks but recalled fewer details both immediately and two days later. These students also felt they put in less effort over time and felt less like the code they submitted was really theirs. The study suggests that while AI tools can improve short-term performance, they might reduce learning and personal connection to the work.
What this means in practice
- •For educational software developers: Design learning tools that combine AI assistance with features requiring students to explain and actively contribute to coding tasks to improve retention and ownership.
- •For online course designers: Create assessments that test not just final code correctness but also recall and understanding to better measure student learning when AI assistance is available.
Authors
Christian Bergh, Benjamin Tag, Alexandra Vassar, Jake Renzella
Abstract
Generative AI can improve students' programming performance, but successful task completion may not reflect what they retain. We examined performance, retention, cognitive load, and ownership in a controlled between-subjects experiment with 59 undergraduate computer science students, 55 were retained for analysis. Participants completed three introductory C programming tasks with access to ChatGPT-4.5 or conventional web search without generative AI. We measured task performance, self-reported mental effort and difficulty, pupillary responses, heart rate variability, and ownership, and assessed cued recall immediately and 48 hours later. ChatGPT-assisted students achieved higher coding scores (89% vs. 69%) but lower recall scores immediately (41% vs. 53%) and after 48 hours (39% vs. 52%). There was no significant difference in the loss of recall information over 48 hours between the groups. Self-reported mental effort increased less across tasks in the ChatGPT condition (Holm-adjusted p = .047), and students attributed less of the submitted code to themselves (45% vs. 81%). Confirmatory physiological tests did not detect significant differences in trajectories between conditions; substantial data loss limits their interpretation. These findings reveal a gap between assisted task performance and subsequent recall and sense of ownership in this setting. They motivate the need for assessment practices and AI learning tools that require students to explain, retrieve, and contribute to the work they submit as active participants in their education.