Large language models struggle to generate cultural kinship terms
Recognized but Not Produced: A Generation Benchmark for Culturally Specific Kinship Terms
Computation and LanguageArtificial Intelligence
Summary
Understanding and using family relationship words correctly in different languages is hard for language AI. The authors tested several large language models with Hindi, Tamil, and Korean kinship terms to see if they could produce and recognize the right words. While models could often select the correct word from a list, they had much more difficulty producing those words on their own. This means that current AI still struggles with culturally specific kinship language, even when the relationships are clearly described.
What this means in practice
- •For multilingual ai developers: Improve training and evaluation of AI systems to better generate culturally specific kinship terms in underrepresented languages.
- •For language technology companies: Enhance language models used in communication apps to correctly produce family relationship terms in diverse linguistic contexts.$Commercial implications: Allows selling AI-powered communication tools with improved cultural and linguistic accuracy in kinship language.
Authors
Sahil Pardasani, Madhusudan Singh
Abstract
Current literature evaluates large language models (LLMs) on multilingual kinship understanding using multiple choice benchmarks, treating it as a recognition problem. We instead prompt five open weight LLMs to generate kinship terms in three non Western languages (Hindi, Tamil, and Korean) across two communicative tasks and pair this with a matched option-supported selection baseline. On identical relation language cells, GPT OSS120B selects the correct term in 90.67% of 75 valid cells but produces an accepted term in 36.00% of the corresponding attempts; Llama 3.370B shows the same pattern (77.92% versus 24.24%). Since the four-option condition displays the candidate terms and does not require script production, the difference is interpreted as an evaluation format gap rather than direct proof that lexical knowledge is intact. On explicitly specified L3 prompts, accuracy varies sharply, from GLM-5.1 at 72.29% to Llama-3.370B at 24.24%. The paternal-lineage advantage is language specific; it is large in Hindi but weak or reversed in Korean, while Tamil shared-term pairs provide a control for measurement variation. These results show that culturally specific kinship generation remains difficult even when the relationship is explicitly stated and motivate generation-based evaluation alongside multiple-choice testing.