Small language models track English grammar mastery better and cheaper
Grammar Concept Annotation at Scale: Deployed Fine-Tuned Small Language Models Outperform Prompted Frontier Models
Computation and LanguageArtificial Intelligence
Summary
Correcting grammar helps people learn languages better, but teachers’ corrections are often hard to organize and track. The authors improved small language models to automatically analyze learner-tutor conversations and identify grammar issues more accurately and much more cheaply than the largest models. They trained a compact model that runs efficiently and matches or beats bigger models in spotting grammar mistakes, helping to monitor language learning progress. Using this model in a real platform led to more engaged learners and increased lesson bookings and revenue.
What this means in practice
- •For language learning platform developers: Integrate efficient small language models to track and assess learner grammar mastery from tutoring sessions at scale.$Commercial implications: Enables scalable commercial language learning products that personalize feedback and progress tracking more affordably than using large model APIs.
- •For educational software engineers: Use fine-tuned small models for automatic grammar concept annotation to improve learner engagement metrics in language apps.
Authors
Marjan Celikik, Ana Peleteiro Ramallo, Javier Morales
Abstract
Corrective feedback is among the best-evidenced drivers of second-language acquisition, yet corrections delivered during lessons rarely accumulate into an actionable view of grammar mastery. Prompted frontier models can provide such a view from learner--tutor lesson transcripts, but they are costly at scale. We close this gap by fine-tuning Qwen3.5 small language models (SLMs) on filtered and rebalanced teacher-generated supervision, then deploying an efficient 0.8B model in an end-to-end grammar mastery tracker for all English learners on our platform. Internalizing the annotation contract into adapter weights enables pairing the 0.8B model with a compact matched prompt rather than verbose instructions. On two human-curated benchmarks, both the deployed 0.8B model and a 4B reference comparator outperform prompted GPT-5.4 and GPT-5.6 Sol in precision and recall under nested matching criteria of increasing strictness: concept, evidence span, and correctness. The deployed 0.8B SLM reduces serving cost by approximately 16$\times$. A feature-level online experiment shows significant gains in learner engagement ($+15.8\%$) and key business metrics, including scheduled hours ($+2.1\%$) and GMV from new lessons ($+13.2\%$).