Summary
Using AI to help students with their work can make it hard to see how much the student is actually thinking versus the AI doing the work. The authors created a six-level system called HADT to measure how much the student controls the AI prompts, independent from the AI's answers. They studied nearly 1,000 AI prompts from first-year software engineering students over ten weeks and found that most students contributed something beyond just asking for answers. The kind of student input changed during different stages of their projects, and the students’ individual differences mattered less than the phase of work. The authors also noticed students rarely gave explicit rules to the AI, possibly because novices don't yet know how.
What this means in practice
- •For software engineering instructors: Use HADT to assess and improve how students engage with AI tools during different project phases in software engineering courses.
- •For educational technology developers: Design AI tutoring and feedback systems that monitor human agency in student prompts to provide better guidance on prompt engineering skills.
Abstract
When students use generative AI to produce their work, the learning process becomes invisible. Established frameworks such as Bloom's taxonomy then classify the cognitive level of what the AI produced, not what the student contributed. We propose the Human Agency Depth Taxonomy (HADT), a preliminary six-tier taxonomy that classifies the engineering contribution a student makes in a prompt, independent of the AI's output. HADT combines three dimensions: contextual bandwidth, engineering abstraction and student agency. We applied HADT to 962 prompts from 105 students' structured AI Journals in an AI-permissive, first-year software engineering course of 145 students, and examined how agency changed over ten journalled weeks. Most prompts (84.2%) carried some student contribution beyond delegation. Agency tracked the development lifecycle. Architecture-discovery prompts dominated the requirements phase (48% of turns) and fell to 5-12% from week 9, while prompts analysing existing systems rose during build and verification (odds ratio 2.88, with turns clustered by student). Two-thirds of the variance in tier lay within students, so agency depended more on the phase of work than on the student. A turn-level rise in delegation in deadline weeks did not hold once turns were clustered by student. Explicitly specifying constraints on the AI's output was rare throughout, which we interpret as a knowledge bottleneck for novices rather than a gap in instruction. We discuss implications for course and assessment design, and for using HADT as a formative lens on how students engage with AI.