Personalized Assessments from Personal Artifacts
Software Engineering
Summary
The authors noticed that with AI tools helping write code, students and professionals might not fully understand the code they produce. To check this, they created a tool called Personalized Probing Puzzles (p³) that gives students short puzzles about their own code. They tested it in a graduate cloud computing class and found it can reveal when students might not fully grasp their code. The puzzles are easy to give and complete quickly, but more research is needed to see how puzzle scores relate to actual code understanding and how to use this method in real code reviews.
Authors
Yufan Zhang, Jaromir Savelka, Seth Copen Goldstein, Majd Sakr
Abstract
The rapid development and popularization of AI-enabled coding agents have meant software engineering students and professionals cannot be assumed to understand their own code, which risks academic integrity and professional accountability. We developed a method called Personalized Probing Puzzles ($p^3$) to evaluate students' understanding of their own code, and tested $p^3$ in a graduate-level cloud computing course. Our pilot study shows that $p^3$ can help identify potential gaps in students' understanding of their own code. The puzzles are automatically generated, asynchronously administered, and finished in minutes. Future work is needed to correlate puzzle results with code understanding and to embed $p^3$ in a professional code review process.