Computing students use large language models mainly to understand concepts and debug
An Empirical Study on how Computing Students Interact with Large Language Models for Learning Purposes
Computers and Society
Summary
Many computing students use large language models (LLMs) like AI chatbots to help with their learning. A study of 225 students found they mostly ask LLMs to explain ideas, learn new things, find information, fix code bugs, and get creative ideas. Students often give detailed questions and try asking multiple times to get better answers. However, students sometimes get wrong or vague replies and have trouble trusting the AI’s explanations. The researchers organized these behaviors using a learning theory to understand how students best use LLMs.
What this means in practice
- •For educational software developers: Design interactive learning tools that incorporate effective prompting strategies and provide trust cues based on how computing students actually use LLMs.
- •For developer tool designers: Improve debugging assistants by integrating common LLM usages and user behaviors identified in computing education contexts.
A survey. It maps existing work.
Authors
Opetunde Ibitoye, Saheed Popoola
Abstract
Large Language Models (LLM) have become popular tools in education. Therefore, understanding how students interact with LLMs for learning purposes is an important aspect of the responsible design and development of LLM based learning tools. However, existing research has focused on evaluating model performance, specific use cases, and students' attitudes toward LLM in education. There has been limited empirical work exploring students' interaction strategies with LLMs in computing education. This study draws on Self Regulated Learning (SRL) theory and conducts a survey of 225 undergraduate and graduate computing students to investigate how students use LLM tools during learning, the perceived learning impact, and the strategies they rely upon when using these tools. Students used LLMs across nine functional categories, most often to explain concepts, learn new material, find information, debug, and generate ideas. They also adopted some strategies such as supplying detailed context and iterative prompting. The most common difficulties encountered with LLM tools were incorrect information, generic responses, and difficulty trusting the model's reasoning. We synthesized these patterns through an SRL lens and offer a Probe Monitor Shape heuristic to map students' LLM interaction strategies onto the phases of self-regulated learning.