Resume screening with AI shows bias from how resumes are presented
Competence-Preserving Resume Perturbations Expose Presentation Sensitivity in LLM Screening
Computation and LanguageArtificial Intelligence
Summary
Resume screening tools that use large language models must decide who is qualified for a job based on resumes. However, these tools can change their decisions if the same candidate’s qualifications are shown in different formats or wording, even when the core information is the same. The authors created controlled tests to measure how stable these decisions are despite changes in resume presentation. They discovered that even strong models often flip their decisions a significant amount of the time when the resume looks different. This means resume screening tools should be tested not only for accuracy but also for consistency.
What this means in practice
- •For human resource technology teams: Identify and reduce decision inconsistencies caused by resume formatting when building automated applicant screening systems.
- •For recruitment platform developers: Create evaluation metrics that assess whether screening models maintain stable candidate rankings across different resume presentations.
Authors
Qiangju Chen, Yang Xiao
Abstract
Resume screeners must infer job-relevant competence from resumes whose presentation can vary substantially in wording, structure, stylistic polish, and document extraction quality. Ideally, such surface variation should not change decisions when the underlying qualification evidence is unchanged. We introduce a controlled audit of this property, constructing occupation-grounded candidate profiles at controlled competence levels and rendering each profile into multiple resume presentations. A deterministic validation gate excludes variants that alter the underlying evidence before scoring. Across six open instruction-tuned LLM conditions, we find a clear disconnect between screening validity and presentation stability. Llama-3.1-8B with its native chat template achieves the strongest validity ($0.781$) yet reverses $29.6\%$ of matched pairwise decisions under competence-preserving presentation changes; Mistral-7B-v0.3 reaches validity $0.644$ with a $41.4\%$ flip rate. Native chat formatting improves validity for several chat-tuned models but does not remove this instability. These results show that resume-screening evaluations should assess not only whether a system identifies stronger candidates, but also whether those decisions remain stable when the same competence evidence is presented differently.