Wiki based course structures improve machine learning tutoring accuracy
Potential for Enhanced Learning in Machine Learning Classes by Using Wiki LLM Indexing
Artificial Intelligence
Summary
Large language models (LLMs) can help students learn by acting as tutors, but they need reliable course materials to give good answers. The authors compared two ways to organize a machine learning course’s content for an LLM tutor: one that searches through a list of stored text snippets and one that builds a linked wiki with clear citations. They found both worked well for simple questions, but the wiki method was much better at answering complex questions that connect ideas from different parts of the course. This wiki also let students check where answers came from, which is helpful when course content changes.
What this means in practice
- •For learning platform developers: Build AI tutors that organize course content into linked wiki pages to improve accuracy on complex, cross-topic questions in technical subjects.
- •For corporate training teams: Structure evolving training materials as dynamic wikis to help AI tools provide up-to-date, traceable answers tied to original sources.
Authors
Brian Wright
Abstract
Large language models are increasingly deployed as course-specific tutors, but their usefulness depends on grounding in vetted instructional materials that are often revised mid-semester. Our prior work built a multimodal retrieval-augmented generation (RAG) system over an authentic machine learning course corpus (Foundations of Machine Learning) and found that retrieval improved contextual grounding, but that fixed retrieval strategies were suboptimal. That motivates a different question: whether how a corpus is structured at ingest time matters more than how much is retrieved at query time. We present a controlled head-to-head comparison of two knowledge representations over an identical classroom corpus: (A) vector RAG, replicating the best-performing configuration from our prior study, and (B) an LLM-compiled wiki (Karpathy framework), in which the corpus is synthesized at ingest into linked concept pages with explicit cross-references and citations back to source materials. We evaluate 59 questions spanning single-fact recall, cross-unit concept linking, synthesis and explanation, and currency after a syllabus revision, scored by an LLM judge against a human-authored rubric. Both representations answered single-fact questions about equally well (9.33 vs. 9.96 of 10), but diverged sharply on questions requiring links across course units. The compiled wiki remained accurate and grounded (9.93; 100% grounded in cited sources), while retrieval scored lower and was markedly less grounded (8.14; 64%). The wiki's citations let students and instructors trace any claim back to the lecture that introduced it, adding a layer of dynamic retrieval that machine learning courses require. While further testing is needed, instructors using AI to support learning in ML courses should consider wiki-based structure for its potential to support foundational elements of best practice.