Understanding Student Perceptions, Mistakes, and Debugging Approaches when Solving Natural Language Programming Tasks
2026-07-06 • Computers and Society
Computers and Society
AI summaryⓘ
The authors studied how beginner programmers use natural language to prompt AI that writes code. They looked at over 900 students in an introductory course to see what common mistakes were made and how students fixed problems when AI-generated code was wrong. They found students often left out important details in their prompts, relying too much on AI to guess them. When fixes were needed, students usually reworded their requests instead of debugging the AI's code directly. Overall, students found prompting easier and more fun than traditional coding.
code-generating AIprompt engineeringnovice programmerscomputational tasksdebugging strategiesnatural language promptsCS1 courseprompt problems
Authors
Victor-Alexandru Pădurean, Kaitlin Riegel, Gweneth Barbre, Musa Blake, Paul Denny, Alkis Gotovos, Juho Leinonen, Stephen MacNeil, James Prather, Adish Singla
Abstract
Learning to communicate with code-generating AI models is an emerging skill for novice programmers. One recent pedagogical approach, Prompt Problems, has students solve computational tasks by writing natural-language prompts for code-generating AI models. However, little is known about the specific prompt-level mistakes novice programmers make, the kinds of computational details they fail to communicate, and what strategies they use to recover when generated code is incorrect. In a CS1 course, we studied attempts by more than 900 students to solve dialogue-based Prompt Problems. We analyzed student reflections, unsuccessful prompts, and reported debugging strategies. Compared to traditional coding tasks, students generally found prompting easier, more enjoyable, and better targeted at developing problem-solving skills. The most common mistakes are related to the omission of key details, suggesting both a failure to acknowledge their importance and over-reliance on AI to infer them. When prompts failed, students focused more on clarifying their intent and reflecting on the provided problem details than on tracing generated code or examining test cases.