Papers for

ai model evaluators

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Generative models trade feature learning against mode collapse risks

The balance between feature learning and collapse in dynamical systems-based generative models

Abstract: Over the past decade, a growing body of work has established that implicit biases in learning dynamics fundamentally shape the solutions found by neural networks, governing their learned features and generalisation performance. However, despite these strides in deep learning theory, far less is known about representation learning in generative models based on dynamical systems, from normalising flows to diffusion. Importantly, empirical evidence suggests that continuous-time generative models can be prone to mode collapse, yet a learning theory explaining such biases remains elusive. To address this, we use tools from dynamical systems and operator theory to express the gradient flow of a general class of stochastic differential equations as a composition of linear operators, allowing us to relate collapse to low-rank biases in generative models. In variational inference and flow-matching tasks, we show that this bias can manifest in two different mechanisms: i) where collapsed models are stable fixed points of the gradient flow, or ii) in sequential feature learning, where smaller modes take progressively longer to be learned, producing an effective collapse for a fixed training budget. Together these findings point to a trade-off between feature learning and collapse, suggesting an operator view as a step towards a unified framework for understanding the learning dynamics of dynamical systems-based generative models.

Mon 9 NovArtificial IntelligenceMachine Learning
The gist
Continuous-time generative models, which create new data by simulating dynamical systems, can sometimes get stuck producing limited results, called mode collapse. The authors used math tools from dynamical systems and operator theory to understand why this collapse happens. They found two main reasons: some collapsed models remain stable under learning, and smaller details take longer to learn, causing an effective collapse when training time is limited. Their work reveals how learning features and avoiding collapse are balanced in these models.
Open → 2611.12156v1

Calibrated readings let users measure neurons in ai models reliably

Calibrated concept profiles: making the reading of a model's neurons a measurement

Abstract: Interpretability has largely moved from reading a model's neurons directly to reading a second model trained on the first: a probe, a sparse autoencoder, a crosscoder. The move answered real faults -- mixed meanings, thresholds without a unit, readings that did not transfer from one study to the next -- but gave up what a direct reading alone provides: nothing trained in between, the neurons themselves named, and a reading that can be repeated and compared. Here the direct reading is made a measurement. A neuron's activation varies with the prompt, so its reading is a distribution over a population of stimuli. We declare that population: every factor around a target construct is drawn at its rate in a reference corpus and logged. From the declared distribution we generate a battery of prompts, and draw it twice. From the battery we compute the calibrated concept profile, one number per neuron, stated in units of the neuron's own spread on ordinary input; the two draws give every number a noise unit. Before the profile is read, a fixed set of checks licenses the battery on the model, and the smallest change the profile can resolve across training checkpoints is measured. Realised for Python on OpenCoder-1.5B and followed across checkpoints spanning two trillion training tokens, the profile gives what the alternatives do not report: a unit on every number, a chance level for every comparison, and which neurons carry a construct at every checkpoint. The battery, the code and the data behind every result are released.

Fri 6 NovComputation and LanguageMachine Learning
The gist
Understanding what individual neurons in AI models represent has been tricky because their activations change with different inputs, and there hasn’t been a consistent way to measure them. This paper introduces a method where the authors define a standard set of inputs to test neurons, turning their activations into meaningful, comparable numbers with known units and noise levels. By doing this repeatedly on a code-writing AI model during training, they show which neurons correspond to specific concepts over time. This approach helps make neuron interpretation more like a precise measurement that others can reproduce and compare.
Open → 2611.09196v1

Large language models cannot reliably report their financial knowledge over time

LLMs Don't Know What They Know: Benchmarking Temporal Self-Knowledge for Financial Backtesting

Abstract: Financial backtests of large language models (LLMs) are meaningful only if the model did not already know the outcomes it is asked to predict. Practitioners guard against this by trusting a vendor's stated knowledge cutoff, by asking the model what it knows, or by instructing it to answer as of a past date. To verify whether any of these safeguards can be trusted, we introduce FinDendro, a benchmark of 2,226 dated U.S. facts from 1999 to 2026, covering monthly macroeconomic series, month-end stock prices and Federal Reserve decisions. Each fact yields four prompts: a value question, Recall and Predict questions that pair each candidate value with a claim of knowing or guessing without a declared date, and a Predict question with a historical declared date. Controls for persistence and dominant outcomes keep chance accuracy near 0.27. Results show that for most frontier models, self-reports understate knowledge. GPT-5 answers 76% of controlled facts correctly but claims to know 38%, and answers 61% of the facts it says it does not know. Knowledge claims can persist beyond the measured knowledge boundary, by up to seven quarters, and a declared date removes most of the claims of prior knowledge while leaving accuracy unchanged. Backtests inside a model's training window therefore cannot be certified leak-free by self-report, instruction or stated cutoff. These findings motivate behavioral measurement of temporal knowledge boundaries, and FinDendro provides a benchmark and procedure for selecting backtest periods based on demonstrated factual knowledge.

Wed 4 NovComputation and Language
The gist
The paper shows that large language models like GPT-5 often do not know when they actually know information about financial events. Models may claim ignorance but still answer correctly, or say they know things even after the known knowledge cutoff dates. The authors created FinDendro, a test with thousands of dated financial facts, to check if models can accurately self-report what they know at different times. They find that self-reports and instructions about knowledge dates are not trustworthy for ensuring predictions are leak-free during backtests.
Open → 2611.06808v1

Changing the judge model alters AI chatbot rankings significantly

Swapping the Judge Changes the Ranking: A Preregistered Study of Cross-Provider Replication in LLM-Judged System Comparisons

Abstract: Practitioners who compare systems with an LLM judge and have no human labels often check a conclusion by repeating it with a judge from another provider. We test how far that check can be trusted. In a preregistered study (with a disclosed post-registration reduction of the judge set), five judge models from three providers re-judged 4,000 Chatbot Arena battles (nine systems, 33 system pairs) that carry human votes, each in both response orders. Agreement with the human vote was low at the verdict level (median Cohen's kappa = 0.22), and system rankings did not survive a change of judge: Kendall's tau with the human ranking ranged from 0.17 to 0.89 (median 0.67), so the preregistered hypothesis that rankings remain stable was not supported. Conclusions replicated by two low-cost judges from different providers matched the human conclusion more often than single-judge conclusions (+0.10 precision), but the gain came from filtering out one poorly aligned judge, and replication involving that judge left a conclusion for only 35% of comparisons. Judges that agreed with humans reached a stable conclusion with no more verdicts than human votes required; the others failed to converge on 26-70% of system pairs at the sample sizes tested (up to 100 verdicts per pair). Disagreements with the human vote co-occurred across judges far more often than independence predicts, although a single human vote per battle cannot separate shared judge bias from voter noise.

Tue 3 NovComputation and LanguageArtificial IntelligenceComputers and Society
The gist
When different AI judge models are used to compare chatbot responses, their decisions often disagree and don’t match human votes well. This means that swapping one AI judge for another can change which chatbot looks best. The authors found that combining two well-aligned AI judges helps make more reliable rankings, but some judges consistently disagree with humans and cause unstable results. They also noticed that judges tend to agree on when to disagree with humans, making it hard to tell if it’s a shared bias or random noise.
Open → 2611.04090v1

Checking test state identity improves vision language action model comparisons

Verifying Test-State Identity in Paired Evaluation of Vision-Language-Action Models

Abstract: Simulation-based VLA evaluation often treats matching task, seed, and episode identifiers as sufficient evidence that two policies faced identical test conditions. Under specific vectorized reset and autoreset conditions, different termination timing can change subsequent initial-state assignment, so the same episode index need not mean the same test state. We present an evaluation-integrity framework for paired comparison - invariants I1-I5, provenance logging, optional scene fingerprints, and an automated checker - and a controlled LeRobot 0.6.1 LIBERO harness study with emulated controllers. Four evidence levels are kept separate: (A) mixed-horizon Sync wave-1 IDs [20, 5, 22, 7] vs [4, 5, 6, 7]; (B) homogeneous separate-job Sync control (16 cells, 0 ID/fingerprint mismatches); (C) Async+Freeze batch-size map differences (12 I3 mismatches; bs=8 incomplete; mechanism unisolated); (D) a synthetic McNemar illustration only. Controllers were emulated and no real-VLA shared-vector experiment was completed. We recommend checking declared/resolved init-state identity and provenance before paired statistics. We do not claim that published VLA leaderboards are generally invalid.

Sat 31 OctRobotics
The gist
Testing AI models that understand images and language while taking actions relies on running them in the same test conditions to compare fairly. This paper shows that just matching test identifiers might not guarantee identical starting states because of how simulations reset. The authors offer a framework that logs and checks detailed setup information to ensure test states truly match before comparing models. This helps make sure comparisons are accurate and fair.
Open → 2611.00896v1

Audio visual benchmark tests evidence based reasoning in omni language models

EvaOmni: An Evidence-Grounded Benchmark for Audio-Visual Reasoning

Abstract: Omni LLMs can naturally locate key audio and visual evidence and integrate information across modalities for audiovisual reasoning. Existing audio-visual reasoning benchmarks that mainly rely on multiple-choice questions usually score models solely based on answer correctness, making it difficult to assess whether a model truly forms a reasoning chain from evidence to the correct answer. In addition, textual or unimodal shortcuts may allow models to answer correctly without analyzing the necessary audio-visual evidence, further weakening the reliability of evaluation benchmarks. To address these issues, we introduce EvaOmni, an evidence-grounded benchmark containing 392 videos and 1,027 purely human-annotated question-answer pairs, covering 15 capabilities and three levels: retrieval and localization, basic understanding, and complex reasoning. We design a series of rigorous annotation procedures in which experts annotate audio and visual evidence and construct video-based distractors; humans review whether questions meet strict inspection criteria, and finally models filter textual and unimodal shortcuts. Beyond high-quality question-answer pairs, we require models to simultaneously return the answer and supporting evidence intervals in order to evaluate the complete reasoning chain. We propose the Coverage Grounded Score (C.G.) to measure how much annotated evidence a model identifies, and the Bottleneck Grounded Score (B.G.) to penalize poor temporal localization in either modality; both are incorporated into answer correctness. Across 7 open-weight and 17 proprietary Omni LLMs, Gemini3.7-Flash achieved the highest overall accuracy (79.4%), C.G. (70.8%), and B.G. (35.1%). At Level 3, it reached 79.1% accuracy, but its B.G. was only 12.3%. These results show that even the most advanced models still have room for development in grounded audio-visual reasoning.

Tue 20 OctComputer Vision and Pattern Recognition
The gist
It’s hard to know if computer programs that answer questions about videos and sounds really understand what they see and hear or are just guessing. The authors created EvaOmni, a careful test containing videos with questions that need finding evidence in both sound and pictures to answer correctly. They made sure the questions and answers rely fully on audio-visual clues and that models must show the exact proof behind their answers. They also introduced scores to check how well models find and pinpoint that proof. Their tests show even the best models still struggle to fully connect their answers to the right evidence.
Open → 2610.28444v1

Teacher advantage in on-policy distillation lies in targeted corrections

Demystifying On-Policy Distillation: Where Does the Teacher's Advantage Live?

Abstract: On-policy distillation (OPD) is increasingly studied as a cheaper alternative to reinforcement learning, yet it remains a form of knowledge distillation (KD) that relies on teacher supervision. We revisit OPD from the KD perspective by asking where the teacher's advantage lies along student-generated trajectories. Controlled token flips at teacher-student disagreements directly measure the accuracy gained from teacher corrections. Single-token flips yield a modest average gain on selected failed trajectories, whereas repeated flips raise accuracy from 28.4% to 74.5%, close to the teacher's 76.9%. Gains continue to accumulate with more flips, but mass- and KL-based selection outperform random flips by 15-18 percentage points at counts of similar magnitude. Selective intervention remains useful after OPD, but greater argmax agreement does not consistently improve accuracy across model pairs. We advocate training schedules that learn more valuable teacher corrections while maintaining distributional stability. Controlled flips and fork-resolution analysis evaluate OPD beyond accuracy by measuring which teacher preferences the student acquires and what further correction can recover.

Tue 13 OctMachine Learning
The gist
Training smaller models to learn from bigger, smarter 'teacher' models can save time, but it's unclear what exactly the smaller models learn from the teachers. This paper studies where the teacher helps most by flipping mistakes in the student's decisions and measuring gains. They find that carefully chosen corrections help a lot more than random ones, and that training should focus on learning these key fixes while keeping stable behavior. Their method also helps evaluate how much students pick up teacher preferences beyond just accuracy.
Open → 2610.19145v1

Hallucination detection labels differ based on faithfulness versus factual correctness

The Labeling Problem in Hallucination Detection Benchmarks: An Empirical Evaluation

Abstract: In recent years, several methods for detecting when large language models (LLMs) hallucinate have been developed. These methods are often benchmarked with open-domain question answering (QA) datasets containing questions and corresponding short reference answers. First, an LLM is used to generate answers to questions within the QA dataset. Then, some automated labeling strategy is used to label these answers as hallucinated or not by comparing them with the reference answers in the dataset. This evaluation setting creates a methodological ambiguity between two criteria: reference faithfulness (whether the answer is fully supported by the reference) and factual correctness (whether the answer is free from contradictions and factually false specific claims). In practice, automated labelers may apply the former criterion even when the intended target is the latter. We study this potential criterion mismatch using 900 human-labeled question-answer pairs spanning three commonly used QA datasets and three generator models, with labels targeting answer-level factual correctness. We evaluate lexical similarity metrics, a reference-entailment NLI baseline, and seven LLM judges under controlled prompt variants as automated labelers. Our experiments reveal substantial disagreement both among automated labeling strategies and between these labels and human annotations. Many strategies also exhibit strong directional error biases, and for most judge-generator pairs, replacing a faithfulness-oriented prompt with a factual-correctness prompt improves agreement with human annotations and reduces false-positive dominance, indicating that automated hallucination labels depend strongly on how the target criterion is specified. Label-source choice should therefore be considered a fundamental part of benchmark design and made explicit, validated, and matched with the benchmark goal.

Tue 6 OctComputation and Language
The gist
Figuring out when AI language models make things up—called hallucinations—is tricky because there are different ways to judge if an answer is wrong. One way is to check if the answer matches a given reference, and another is to see if the answer is factually true regardless of that reference. The authors found that automatic systems often get confused between these two and label answers inconsistently compared to human judgment. They showed that changing how the question is asked to AI can improve agreement with humans. This means careful design and clarity about what kind of error is being labeled are very important.
Open → 2610.08026v1

Open contrastive decision model performs near chance on judge benchmarks

CLM-as-a-Judge: Evaluating an Open Contrastive Decision Model on Public Judge Benchmarks

Abstract: An open contrastive decision model is near chance as a judge on the hard public benchmarks: Contrastive-LM/CLM-v0.1-8B scores between 0.351 (best- of-four, chance 0.250) and 0.593 (pairwise, chance 0.500), is statistically indistinguishable from coin flipping on RM-Bench and JudgeBench, and answers every HaluEval item with one constant label, matching the trivial always-first baseline at 0.581. Judges with the same parameter count score far higher everywhere: a reward model reaches 0.764 to 0.976 and a generative judge 0.611 to 0.778, and every gap to CLM is significant after Benjamini-Hochberg correction. Two properties do work. Raw confidences are overconfident by up to +0.401, yet one pooled temperature fit on held-out calibration items repairs expected calibration error to at most 0.062, and the repaired confidence ranks the model's own errors above chance on three of six benchmarks. The decision order-flip rate is 0.0002 against 0.2188 for the generative judge, and the length-preference shift is -0.023 against -0.217. The confidence-gated cascade, however, escalates between 0.923 and 1.000 of items to the strong judge at the preregistered 0.97 retention bar: calibrated confidence about a near-chance judge has almost nothing to keep. The design: five public preference benchmarks and one hallucination benchmark with real labels, scored under a preregistration frozen before any test item was seen, against generative, reward-model, and trivial baselines, with per-item predictions released.

Mon 5 OctMachine LearningArtificial IntelligenceComputation and Language
The gist
This paper studies how well a type of AI model called an open contrastive decision model works as a judge on difficult tests. The authors found that this model performs about as well as random guessing on several public benchmarks. Other similar models with the same size perform much better at judging. The paper also shows that while the model’s confidence scores are too high, calibrating them can improve how well confidence indicates errors. However, using these confidence scores to decide when to rely on stronger judges doesn’t work well because the model rarely has confident enough answers.
Open → 2610.07177v1

SOL measures how closely text generation matches data distributions

SOL: Measuring Gaps between Text Distributions by Double Sliced Wasserstein Metrics

Abstract: Evaluating text generation requires measuring how well the generated distribution matches the data distribution. For autoregressive models, this is done by the perplexity. Diffusion and flow-based language models can only provide a likelihood bound, whose tightness differs between model families. Sample-based substitutes such as generative perplexity with entropy do not consider the distribution fit. We propose SOL, a distance between text distributions. Each sequence is represented by the empirical measure of its hidden states under a fixed transformer and the distributions of these measures are compared by the double sliced Wasserstein distance. We prove that SOL is a metric if the transformer is injective. Experiments show that SOL detects distributional failures, recovers expected model trends, and provides stable sample-based estimates. We put forward SOL to fill the gap in the current evaluation protocol used for non auto-regressive models. As a first step we use SOL to re-evaluate a variety of models trained on OpenWebText.

Mon 5 OctComputation and LanguageMachine Learning
The gist
Evaluating how well computers generate text depends on comparing the texts they create to real examples. Traditional methods work well for some models but not for newer ones like diffusion or flow-based language models. The authors propose SOL, a new way to measure differences between text sets by looking at hidden patterns inside a fixed transformer model. Their method can spot when generated text differs from real data and provides stable ways to compare different models. They used this approach to reassess several language models trained on a popular dataset.
Open → 2610.06513v1

Instruction following model scores vary widely with wording changes

When Verifiable Counts Depend on Wording: Auditing Wording Robustness in Instruction Following

Abstract: Verifiable instruction-following benchmarks often express each constraint through one fixed template. We test whether scores remain stable when the operational requirement is unchanged but its wording varies. We introduce WISE, a matched evaluation suite and reporting protocol instantiated on exact word count, keyword inclusion exactly once, and an inclusive 8--12 word range. Across 100 matched tasks, up to thirteen models from seven providers, and repeated generations scored over the complete visible output, wording alone produces substantial compliance shifts. In an avoidance-family panel, five avoidance and exclusion forms fall below the positive baseline, while constructional controls also shift compliance substantially: in the nine-model control panel, compliance is 54.9% for the original positive form, 48.2% for a longer positive form, 36.7% when the target appears later, and 33.8% for AVOID1. A strict JSON-structure probe shows wording sensitivity beyond counting, with a different direction of effect. Effect sizes, failure directions, weakest forms, and model rankings vary across realizations. Under the most disruptive exclusion form, the top-ranked model changes and 24.1% of strictly ordered model pairs reverse. Human validation further shows that unanimous agreement on an exact-count interpretation can coexist with substantially different model behavior. WISE supplements conventional scores with mean and worst-form compliance, wording gaps, failure profiles, and ranking stability.

Sun 4 OctComputation and LanguageArtificial Intelligence
The gist
When AI models are tested to see if they follow instructions correctly, their success can change a lot depending on exactly how the instructions are worded—even if the meaning stays the same. The authors found that small differences in wording can lead to big differences in how well models seem to comply. They created a new test called WISE that compares many wording variations to better understand how wording affects scores. This helps show that current evaluation methods might miss important details about how models really follow instructions.
Open → 2610.05278v1

Function level scores measure flag rate more than vulnerability detection quality

A Function-Level Vulnerability Score Measures Flag Rate More Than the Model: Protocol Effects on Paired Benchmarks

Abstract: Language models are increasingly evaluated as vulnerability detectors, and scores reported for similar models differ widely between papers. We measured how much of that difference evaluation protocol accounts for, with model outputs held fixed. In a paired test, a model must flag a vulnerable function and clear its version after a fixing commit. Three choices that published evaluations make differently were varied one at a time: metric, verdict extraction and output budget. Seven frontier and large open models were evaluated on five released pair benchmarks and a set pooled for this work under one protocol, and 61 open models of 1.5B to 36B parameters on the pooled set. Function-level F1 follows how often a model flags both functions of a pair (Spearman $+0.86$ over 42 combinations) and is nearly unrelated to pair-level correctness ($+0.16$). On the pair score, extraction changes a model's number by $+0.001$ at the median and budget by $+0.02$ with an interval through zero, whereas the model changes a benchmark's number by up to 0.18 and the benchmark a model's by up to 0.16; a function-level score therefore measures flag rate more than model. For 37 of 68 models the difference between correct and reversed pairs is within its 95% interval of zero, the value for a null model that flags each function at a fixed rate, while both-flagged and both-cleared rates exceed that null by 0.055 on median, and for 64 of 68 both functions of a pair receive one answer more often than independence predicts: verdicts are determined by the text common to both functions. On length-matched pairs a linear probe on activations separates 0.78 by within-pair ranking, against 0.64 for a tf-idf baseline and 0.5 for length; the generated verdict is near chance for three of six models and at 0.55 to 0.57 for the other three, and a prompted logit is at chance for all six.

Sat 26 SeptMachine Learning
The gist
Detecting software vulnerabilities using AI language models involves checking if the model flags functions as risky and then clears them after fixes. The authors found that the way these models are evaluated affects the reported scores more than actual model differences. They show many scores mostly reflect how often the model flags code rather than how accurately it detects vulnerabilities. Also, judgments about paired code snippets tend to depend on shared text rather than deeper understanding.
Open → 2609.32890v1

Emoji rating bias hides true differences among language models

Two Emojis of Difference: What Multilingual Affective Generation Benchmarks Actually Measure

Abstract: We audit a multilingual affective generation benchmark eight instruction-tuned LLMs producing emoji summaries for 17,100 Bangla, English and Hindi sentences, with 6,960 human judgements and find its headline conclusions to be artefacts of the measurement instrument rather than properties of the systems. Treating annotators as a random rather than a fixed factor, no system differs significantly from any other ($F(7,14)=0.59$, $p=0.76$), although the conventional analysis declares 19 of 28 pairwise differences significant. Annotator identity explains far more rating variance than system identity, and the winning system changes whenever any single annotator is removed. The ordering that does emerge tracks output length: mean emoji count explains 78.7\% of between-system variance, and a within-item length-matched comparison over 2,599 pairs reverses the leaderboard. We further show that cross-provider anisotropy differences vanish under mean-centring, that per-language token costs change sign with the normalising unit, and that multi-view row-wise splits inflate macro-F1 by $3.1$ points and change the top-ranked system. In place of preference scoring we propose **emoji-affect decodability**, a reference-based probe whose rankings are stable to $\pm0.003$ macro-F1 across seeds.

Thu 24 SeptComputation and Language
The gist
Measuring how well language models generate emotional emoji summaries in multiple languages is tricky because human ratings vary a lot depending on the reviewer. The authors show that differences between language models mostly disappear when factoring in individual annotator preferences. They also find the number of emojis used affects scores more than quality. Instead of current rating methods, the authors propose a more stable way to measure emoji-based emotion generation.
Open → 2609.29445v1

Llm judge saves cost by escalating unsure cases for review

JEV-as-a-Judge: Accept When Confident, Escalate When Unsure

Abstract: LLM-as-a-judge enables evaluation across diverse tasks, but inference cost and confidence reliability become critical at scale. We study whether a decision-only judge can provide an economical first pass and identify when stronger evaluation is needed. Comparing jev-as-a-judge with sixteen generative and reward-model judges, with blinded human adjudication, we find it within three percentage points of a state-of-the-art LLM judge, our strongest comparator, on ordinary preference and evidence-grounded factuality at 0.36% of the comparator's fee. Larger gaps arise when judgments require checking a derivation or resisting an elaborately written wrong answer. On several benchmarks, JEV's gap to this comparator is concentrated in low-confidence decisions. A frozen cascade that accepts confident verdicts and escalates uncertain ones retains 99% of the comparator's accuracy at lower cost.

Tue 22 SeptArtificial Intelligence
The gist
Checking if language models can decide which answers are right or wrong can be expensive and sometimes unsure. The authors studied a simple model that only gives a yes or no answer cheaply and raises tricky questions to a more expensive system. They found this two-step plan is almost as accurate but much cheaper, as it only asks for help when unsure. This approach can make evaluating many answers faster and cost less.
Open → 2609.26550v1

De-identified résumés still reveal ethnicity beyond language details

Beyond the Name: Demographic Leakage in De-Identified Résumés and Evaluation Artifacts in LLM Bias Audits

Abstract: De-identified résumé screening assumes that redacting explicit fields prevents ethnocultural inference; however, recent audits attribute residual leakage to declared languages. We investigate whether eliminating language fields resolves this leakage across nine open-weight models and 620 counterfactual résumés. By holding language attributes strictly identical, we isolate unstructured prose across five ethnocultural conditions and three cue-salience tiers. Target-group recovery averages 0.757 overall and saturates at 1.000 under high salience, demonstrating that non-language prose sustains demographic inference. Crucially, models diverge only under faint cues (0.086-0.690), establishing salience as an essential evaluation axis. Furthermore, pairwise LLM-as-a-judge outcomes are highly sensitive to evaluation design: forbidding ties yields an apparent selection-rate ratio of 0.39 alongside strong position and content effects, whereas permitting ties produces near-universal ties for most models ($\ge94\%$). Downstream scoring shows only very small between-condition differences, highlighting the need to distinguish demographic signals recoverable from résumé content from effects introduced by the evaluation protocol.

Tue 15 SeptComputation and LanguageArtificial Intelligence
The gist
Removing obvious personal information like language skills from résumés does not fully stop AI models from guessing a person’s ethnic background. The authors tested multiple AI models on résumés where they controlled language details to be the same but changed other writing aspects. Even without language clues, the AI could often identify ethnic groups, especially when subtle hints in the text were strong. They also found that how evaluation tests are designed can greatly change the results, showing that measuring bias needs careful methods.
Open → 2609.16501v1