CreateScore cuts AI costs and boosts resume screening accuracy
CreateScore: Domain-Theory-Informed Bayesian Routing for LLM-Based CV Screening
Artificial Intelligence
Summary
Screening many resumes with powerful language AI models can be expensive. The authors created CreateScore, a system that uses a smaller AI model for simple decisions and only calls a bigger AI when unsure. This method saved a lot of computing resources and improved decision accuracy compared to using only the smaller model. However, the system couldn’t perfectly identify which decisions needed the bigger AI and is meant as a tool to reduce costs, not to replace human hiring teams.
What this means in practice
- •For human resources tech teams: Implement cost-saving AI resume screening by routing simple cases to smaller models and escalating uncertain ones to larger models.
- •For legal compliance teams: Audit AI-assisted hiring tools that combine uncertain decision escalation with interpretable Bayesian scoring graphs to ensure cost effectiveness and transparency.
Authors
Rupsa Roy
Abstract
Large language models (LLMs) can support rubric-based screening of CVs, but applying a high-capability model to every candidate and criterion is costly. We present CreateScore, a domain-theory-informed Bayesian network for criterion-level LLM routing. A hand-specified directed acyclic graph with Dirichlet-multinomial conditional probability tables converts CV evidence into posterior uncertainty; low-uncertainty decisions are resolved by a local 8B model and uncertain ones are escalated to a 120B reference model. The graph is causally motivated, but the system performs standard Bayesian conditioning, not causal inference. The escalation threshold is calibrated on a training fold (target: 70% resolved locally) and then frozen. On 200 synthetic Data Science CVs (139 training and 61 test candidates, five criteria), 77.7% of criterion decisions were resolved locally (237 of 305). Relative to a reference condition in which the 120B model adjudicated every criterion, routed escalation reduced token use by 65.2% and raised exact score agreement from 32.8% (8B alone) to 42.6% (95% CI 31.0-55.1%); at n = 61 the gain was not statistically distinguishable. The uncertainty signal did not, however, identify the decisions on which the 8B model erred: disagreement with the reference was 16.2% among escalated and 19.4% among locally resolved decisions (AUROC 0.47, 95% CI 0.39-0.56), no better than random selection. We also document how an earlier evaluation was invalidated when truncated reasoning-model outputs were silently replaced by local labels, and we recommend safeguards for cascade evaluation. CreateScore is supported as an auditable cost-reduction mechanism, not yet as a targeted error detector, and is not an autonomous hiring system.