Revisiting the Performance of Generative Artificial Intelligence on Introductory Object-Oriented Programming Assessments: Insights from 2026
2026-08-17 • Software Engineering
Software EngineeringArtificial IntelligencePerformance
AI summaryⓘ
The authors tested five popular AI programs to see how well they can solve real beginner programming problems in an object-oriented style. They found that these AI systems generally did better than most students and often got perfect scores on longer tasks. However, the AI sometimes made mistakes like producing code that wouldn't run and had trouble with some tricky concepts like interfaces and inheritance. Compared to last year, these systems have improved but still show common errors. The study helps understand how AI can be used responsibly in programming education.
Generative AILarge Language ModelsObject-Oriented ProgrammingInterfacesAbstract ClassesInheritanceCode CompilationProgramming AssessmentAI-assisted ProgrammingEducational Technology
Authors
Marina Lepp, Joosep Kaimre
Abstract
Recent advances in Generative Artificial Intelligence (GenAI) have substantially improved the ability of large language models (LLMs) to generate and explain source code. However, their performance on authentic object-oriented programming (OOP) assessments remains insufficiently understood. This study evaluates five widely used GenAI systems, ChatGPT-5.2, DeepSeek-V3, Gemini 2.5 Flash, Claude Sonnet 4.5, and M365 Copilot, using programming tests and examination tasks from an introductory university OOP course. The generated solutions were assessed using the same grading criteria applied to students and compared with historical student results from the same course, as well as findings from the previous year. Common errors were also analyzed to identify recurring limitations across models. All evaluated GenAI systems achieved higher scores than the average student cohort and frequently obtained full marks on longer programming tasks. Nevertheless, they occasionally produced non-compiling code and continued to struggle with advanced OOP concepts, particularly interfaces, abstract classes, and certain inheritance-related tasks. Performance was also limited on graphics-related questions involving image interpretation. Compared with the previous year, the evaluated systems demonstrated noticeable improvements across most assessments while exhibiting several recurring error patterns. The findings provide an updated evaluation of the capabilities and limitations of contemporary GenAI systems on authentic introductory OOP assessments. They also offer evidence that can inform the design of programming assessments, the responsible integration of GenAI tools into software engineering education, and future studies evaluating the evolution of AI-assisted programming.