Confidence Without Competence in AI-Assisted Knowledge Work
2026-04-10 • Human-Computer Interaction
Human-Computer Interaction
AI summaryⓘ
The authors studied how different ways of interacting with Large Language Models (LLMs) affect student learning and thinking. They designed a system called Deep3 with three interaction styles: future-self explanations, contrastive learning, and guided hints. Their tests showed that while simple answers made students feel they understood more, they actually learned less. Future-self explanations made students think more and better matched their confidence with actual understanding, but were harder to use. Guided hints helped students learn the most without making them too frustrated. This shows how effort, confidence, and learning can be different when using LLMs for studying.
Large Language Modelscognitive workloadfuture-self explanationscontrastive learningguided hintsperceived understandingobjective learningoverconfidencestudent learninginteraction design
Authors
Elena Eleftheriou, George Pallis, Marios Constantinides
Abstract
Large Language Models (LLMs) are widely used by students, yet their tendency to provide fast and complete answers may discourage reflection and foster overconfidence. We examined how alternative LLM interaction designs support deeper thinking without excessively increasing cognitive burden. We conducted a two-phase mixed-methods study. In Phase 1, interviews with 16 Gen Z students informed the design of Deep3, a web-based system with three interaction modes: \emph{a)} future-self explanations, \emph{b)} contrastive learning, and \emph{c)} guided hints. In Phase 2, we evaluated Deep3 with 85 participants across two learning tasks. We found that a standard single-agent baseline produced high perceived understanding despite the lowest objective learning. In contrast, future-self explanations imposed higher cognitive workload yet yielded the closest alignment between perceived and actual understanding, while guided hints achieved the largest learning gains without a proportional increase in frustration. These findings show that effort, confidence, and learning systematically diverge in LLM-supported work.