Letter positions help identify complex french words from simple ones
Using Letter Positional Probabilities to Assess French Word Complexity
Computation and Language
Summary
Some French words are harder to recognize than others. The authors studied many words and how long it took people to recognize them, using this as a measure of complexity. They looked at the letters in each word and found patterns, especially in the positions of letters and accents, that relate to whether a word is simple or complex. They showed that just knowing word length and letters in the first few positions can predict complexity fairly well.
What this means in practice
- •For natural language processing engineers: Improve French word recognition models by incorporating letter position patterns linked to word complexity measured by reaction times.
- •For text-to-speech developers: Use letter position and accent information to better identify complex French words, enhancing pronunciation and pacing in speech synthesis.
Authors
Michael Dalvean
Abstract
We use Letter Positional Probabilities (LPPs) to examine the orthographic differences between simple and complex French words. We draw from a sample of 38,000 French words scored by reaction time on a lexical decision task. We take the 5000 highest reaction time words and the 5000 lowest as a proxy for high and low word complexity groups respectively. We create 18 variables based on the structure of the words, such as length and number of syllables. We also reduce each word to a string of binary values by one-hot encoding each letter according to the 42 characters of the standard French alphabet up to a length of 25 letters. Thus, each word is represented by 42 x 25 binary characters. We then create LPPs by looking at the average binary scores for simple as opposed to complex words for all letter/positions. We find that there is a high number of LPPs that are associated with word complexity. For example, we find that most accented letters are associated with simpler words. We examine the effects of structural and LPP variables in classifiers and find that word length alone enables a classification accuracy of 77.27%, while adding the binary variables from the first 6 letter positions provided an accuracy of 83.95%. We discuss possible explanations as to how complexity could be encoded in orthography.