Language Proficiency Assessment from Eye Movements in Naturalistic Passage Reading
2026-08-31 • Computation and Language
Computation and Language
AI summaryⓘ
The authors tested a new way to measure language skills by tracking how people’s eyes move when they read passages in English as a second language, instead of traditional tests like vocabulary or grammar quizzes. They showed this eye movement method works well even with longer texts and different reading goals. They also found that the scores can be biased if a person’s first language is similar to English, but developed a fix for this problem. Finally, their method was more consistent compared to usual language tests. This study supports using eye tracking as a reliable tool for language proficiency assessment.
language proficiency testingeye movement trackingsecond language acquisitionreading comprehensionbehavioral tracesnative language influencescore debiasingtest reliabilitylinguistic biascognitive assessment
Authors
Shachar Frenkel, Ido Falah, Omer Shubi, Yevgeni Berzak
Abstract
Standard language proficiency tests rely on linguistic tasks such as vocabulary, grammar and reading comprehension quizzes. An alternative, cognitively motivated approach, introduced in Berzak et al. (2018), proposed instead to predict language proficiency from behavioral traces of eye movements in reading. In this work, we validate and extend this approach from single sentences to more naturalistic reading of contextualized passages in English as a second language, new proficiency measures, prediction models, and reading in an information seeking regime. We find that the approach is effective in all these evaluations. We further address two key open questions on eye movement based proficiency testing: (1) potential scoring biases that reflect the proximity of the reader's native language to English, which may undermine validity, and (2) its reliability. We find that eye movement based proficiency scores are indeed biased towards L1s that are linguistically closer to English. We propose a score debiasing method which effectively remedies this issue. The reliability analyses suggest that eye movement proficiency scores are more reliable than standard language proficiency scores. Overall, our results strengthen and broaden the empirical foundations for future eye movement based language assessment technologies.