Student self-reports accurately track programming skills with specific exceptions
Measuring Student Self-Assessment against Viva-Demonstrated Mastery in a Large First-Year Programming Course
Human-Computer Interaction
Summary
Students often rate how well they understand programming tasks themselves, but the accuracy of these ratings can vary. This study checked how well these self-assessments matched up with oral exams on individual programming tasks for first-year students. It found that most self-reports were very accurate, but a few students made mistakes mainly on harder tasks. The findings suggest self-assessments are useful but can be improved by focusing feedback on certain students and tasks.
What this means in practice
- •For educational software developers: Design dashboards that verify and focus feedback on task areas where students tend to overestimate their programming mastery.
- •For corporate training teams: Use task-level self-assessment verification to improve accuracy in employee skill tracking for programming-related training modules.
Authors
Sakshi Sharma, Pavani Ayinampudi, Aditya B. M. V., Jinal Gupta, Prakash Hegade, Rohit Sharma, Meenakshi V, S. R. S. Iyengar
Abstract
Mastery-based education increasingly places the reporting of learning progress in students' hands, who record task completion on learning dashboards. The usefulness of such self-reports depends on how closely reported mastery corresponds to demonstrated competence. Most evidence on student self-assessment compares an overall self-rating with an overall examination score and therefore provides limited evidence about which tasks or which students account for the mismatch. This study examines first-year students' self-assessment against viva-demonstrated mastery at the level of individual tasks across a ladder of sixty programming tasks. The study draws on a large first-year programming course taught in 2023, involving 203 students and 12 examiners, in which every reported task was verified through an oral viva. Because a task entered the Viva only after it was reported, the design is one-sided and captures over-estimation but not under-estimation. Of 11,093 reported tasks, 10,885 (98.1%) were demonstrated, indicating a high degree of correspondence between self-report and demonstrated mastery. The remaining 208 overestimations were not evenly distributed. A small number of students accounted for most of the errors, and they occurred mainly on difficult tasks near the end of the task ladder rather than on higher-point tasks. This task-level analysis shows that high overall self-assessment accuracy can coexist with specific areas where reported and demonstrated mastery diverge. It also provides a practical basis for directing additional verification and formative feedback toward students and tasks where such divergence is more likely.