Larger context windows help correct grammar with fewer errors
Larger Context Window, Fewer Overcorrections: Optimizing Prompts and Batching for Minimal-Edit Grammatical Error Correction
Computation and Language
Summary
Correcting grammar mistakes automatically can sometimes make too many unnecessary changes to sentences that are already correct. The authors show how giving language models more sentences to look at together helps them avoid overcorrecting. They also created special instructions that tell the model exactly which grammar mistakes to fix. Combining these ideas, their method nearly matches the accuracy of fully trained models without needing extra training.
What this means in practice
- •For language technology developers: Improve grammar correction features in text editors by reducing unnecessary edits using larger batch inputs and targeted prompt instructions.
- •For customer support teams: Enhance automated grammar checks in customer communications to minimize rewriting well-formed messages, improving clarity and preserving tone.
Authors
Kateryna Karpo, Artem Chernodub
Abstract
Minimal-edit Grammatical Error Correction (GEC) is a challenging task for zero- and few-shot prompted Large Language Models (LLMs), which systematically overcorrect and degrade $F_{0.5}$ by rewriting well-formed spans. While fine-tuning provides an effective solution, it imposes substantial infrastructure demands. We introduce a prompt-based approach that closes the gap to fine-tuned models through three advances in GEC prompting methodology. First, we introduce taxonomy-based instructions to enforce minimal-edit constraints with a comprehensive list of grammatical error rules, equipping the LLM with a bounded, metric-aligned scope of correctable edits, which benefits the strongest models while remaining model-dependent overall. Second, we show that batching multiple uncorrected sentences into a single input context acts as a targeted regularizer against overcorrection, systematically reducing the edit rate across diverse LLM families; we hypothesize this arises from attention dilution effect induced by the bounded capacity of self-attention scores. Finally, LLM-assisted Prompt Optimization refines these instructions. Powered by Gemini 3.1-Pro, our prompt achieves $F_{0.5}=78.32$ on the BEA-2019 test set - establishing a new prompt-based SOTA while shrinking the gap to the fine-tuned single-model SOTA (Staruch et al., 2025) to a mere $0.38$ points. Code, prompts, and outputs are publicly available.