Language in job posts triggers gender and racial bias in hiring ai
Linguistic Triggers of Gender and Racial Bias in Open-Weight LLMs Applied to Recruitment
Computation and LanguageArtificial IntelligenceComputers and Society
Summary
Language used in job postings can cause large language models (LLMs) to show unfair bias when recommending candidates. The authors studied six open LLMs and found words that sound agentic (like confident or assertive) made female candidates less likely to be recommended. Similarly, coded language that excludes certain groups reduced scores for non-White candidates and discouraged them from showing interest. The study offers a way to check these biases before using LLMs in hiring to meet legal and ethical standards.
What this means in practice
- •For hr software developers: Implement bias detection tools that identify problematic job post language to reduce gender and racial bias in AI-driven candidate recommendations.
- •For regulatory compliance teams: Use the provided audit protocol to evaluate recruitment AI models for discriminatory risks to ensure adherence to legal standards like the EU AI Act.
Authors
Kosuke Kitahara, Nobuhiro Yamaguchi
Abstract
Open-weight large language models are rapidly entering hiring pipelines, yet their discriminatory failure modes -- and the regulatory exposure these create under the EU AI Act high-risk classification (Annex III) and U.S. EEOC adverse-impact analysis -- remain poorly understood. We present the first systematic, multi-model audit of open-weight LLMs that treats job-posting language as the primary experimental variable, evaluating six models (Llama 3.2, Mistral, Gemma 3, Qwen 3, Phi 3, DeepSeek-R1) across four controlled experiments that jointly probe recruiter-simulation and job-seeker-simulation tasks. We find that (1) agentic posting language depresses recruiter recommendation scores for female candidates (r_rb = 0.309, p_Bonf = 7x10^-5; model-fixed-effects r_rb = 0.448), while communal language partially reverses the penalty; and (2) coded-exclusion language suppresses non-White recruiter scores at large effect sizes (r_rb = 0.646-0.758) and, on the job-seeker side, selectively deters non-White personas from expressing interest -- operationalizing a chilling-effect mechanism at scale. A label-ablation experiment isolates the explicit demographic persona label as the primary causal driver, and Word Embedding Association Tests corroborate these findings at the representational level (d = 1.01-1.45 under Caliskan et al.'s multi-word gender attribute lists). We translate these results into a concrete pre-deployment audit protocol -- posting-vocabulary scoring, persona-conditioned LLM probing, and adverse-impact flagging against the four-fifths threshold -- that operationalizes the documentation and risk-management obligations Annex III imposes on high-risk AI in recruitment.