Chinese benchmark evaluates AI models for recruitment tasks
EHR-Bench: A Chinese Multi-Task Benchmark for Recruitment AI
Computation and LanguageInformation Retrieval
Summary
Recruitment involves many different steps, like writing job posts, sorting candidate profiles, and matching people to jobs. The authors created EHR-Bench, a test set in Chinese that includes many recruitment-related tasks to see how well AI models perform across these areas. They gathered thousands of examples and tested twelve AI models, finding that each model is better at some tasks than others. This benchmark helps compare models fairly and pick the right AI tools for recruitment jobs.
What this means in practice
- •For recruitment software developers: Evaluate and select AI models tailored to specific recruitment tasks like candidate ranking and job description drafting using a comprehensive Chinese-language benchmark.
- •For human resources analytics teams: Systematically assess AI tools on multiple recruitment operations to improve matching quality and operational workflows in Chinese recruitment contexts.
Authors
Ruijia Yang, Bing Xu, Zulong Chen, Zhipeng Xu, Jia Xu, Yipeng Yu, Ying Zhou, Zeyi Wen
Abstract
Recruitment applications span diverse operations, from drafting job descriptions and interpreting queries to organizing profiles, assessing fit, and recommending candidates and jobs. Systematic LLM evaluation therefore requires task coverage that reflects this range of business needs. We introduce EHR-Bench, an open Chinese benchmark comprising twelve tasks across six capability families and five recruitment scenarios. The tasks cover generation, extraction, classification, retrieval, semantic similarity, and reciprocal ranking within one evaluation framework. An internal recruitment dataset guides task definitions, the occupation taxonomy and sampling targets, output requirements, and scoring protocols. Following these specifications, we construct 8,336 task instances from public materials and controlled augmentation, with source-aware expert review and documented provenance. We evaluate twelve LLMs through task-level, capability-level, and aggregate comparisons. Closely ranked models exhibit distinct task strengths, and performance varies across input sources, reference evidence, and candidate pools. Five shared models retain the same observed Overall ordering on a private operational reference, with variation in capability and task correspondence. EHR-Bench provides a reproducible foundation for systematic model assessment, task-specific model selection, and further research on LLMs for recruitment. The project repository is available at https://github.com/EhrBench/EHR-Bench.