Small language models guided to teach grammar in language learning
Grammar Control for Language Learning with Sequential Monte Carlo and Small Language Models
Computation and Language
Summary
Learning a new language is easier when learners get practice with just the right grammar. The authors found a way to guide small, simple language models to produce grammar learners need, using a technique called sequential Monte Carlo. This approach lets these smaller models perform as well or better than much bigger models without needing tons of extra training data. They also showed how to create new grammar guides automatically, making it work for many grammar points without extra effort.
What this means in practice
- •For mobile app developers: Build language learning apps that deliver grammar-targeted practice using efficient small models running directly on devices.$Commercial implications: Enables on-device language learning apps with strong grammar control without needing large cloud models, improving speed and privacy.
- •For dialogue system engineers: Incorporate grammar-focused response steering into small-scale conversational agents to improve language instruction quality.
Authors
Kate McCurdy, Jacob Hoover Vigly, Thomas Hikaru Clark, Karolina Stańczak, Yahya Emara, Joshua B. Tenenbaum, Alexander K. Lew, Ryan Cotterell, Timothy J. O'Donnell
Abstract
Second language learners benefit from exposure to level-appropriate grammar in interactive language use. While prompted frontier language models can usually produce target grammar structures, less resource-intensive models that can be run on device often fail in this task. We use sequential Monte Carlo (SMC) to steer dialogue turn generation in small lan- language models toward target grammar constructions from an expert-developed pedagogical resource. With a minimal set of hand-coded potential functions, SMC achieves near-perfect coverage of target grammar, enabling small models to outperform state-of-the-art results from models 140 times larger. We also use code synthesis with frontier models to generate new potentials, showing that our method scales to hundreds of target grammar skills with no need for additional training or supervised data.