Papers for

benchmark developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Prompt injection attacks vary widely across nine key differences

The Devil is in the Context: Nine Dimensions of Prompt Injection Attacks

Abstract: Prompt injection is usually discussed and evaluated as a single, homogeneous type of attack. However, vulnerabilities reported in real-world systems show that prompt injections vary widely. Moreover, the same instruction can be a prompt injection in one context and a legitimate request in another, depending on the user's intent and the system's policies. Ignoring this variation leaves gaps in how defenses are evaluated, and can give a false sense of security against real-world attacks. We identify nine dimensions along which prompt injection attacks differ. Together, they form a taxonomy that describes the injected instruction itself, its relation to the user's task, and how it manifests in the target system. Each dimension affects which defenses can detect or prevent an attack. We first use the taxonomy to characterize six real-world prompt injection vulnerabilities, showing how much these attacks differ. We then apply it to nine prompt injection benchmarks and find that most cover only a small region of this space. Combining benchmarks helps, but 13% of pairwise category combinations remain untested, mostly among the less commonly represented categories. Finally, we show that the taxonomy supports automated annotation of benchmarks and targeted generation of new test cases to fill these gaps. The taxonomy is also useful beyond benchmarks. Defense authors can use it to state which types of prompt injection they evaluated against, and system operators can use it to identify which attacks apply to their system and choose defenses to match.

Tue 3 NovCryptography and Security
The gist
Prompt injection attacks happen when someone tricks an AI system by sending malicious instructions that look normal but change how the AI behaves. This paper shows that these attacks are not all the same, but differ in nine important ways, including what the instruction says and how it fits with the user's goal. The authors studied real examples and common tests, finding many types of attacks are not covered well. They also created a way to label and find gaps in current tests, helping improve defenses and better match protections to attack types.
Open → 2611.04672v1

Nested recursion operators show no clear gains in reasoning models

Nested Shared-Operator Recursion on ARC-AGI-1: An Empirical Study of HRM, TRM and URM

Abstract: Recursive reasoning models refine persistent internal states by repeatedly applying learned operators. We investigate whether adding a persistent intermediate state that integrates repeated updates of a faster state through the same operator improves predictive performance relative to each model family's baseline at matched parameter counts. We define this organisation operationally and implement it within three architectures: the Hierarchical Reasoning Model (HRM), Tiny Recursive Model (TRM) and Universal Reasoning Model (URM). Using the official codebases on ARC-AGI-1, we compare each baseline with its parameter-matched nested variant. We evaluate held-out queries from tasks whose demonstrations were available during training. The experiments use a single-GPU budget far below the published training protocols. Under the tested conditions, nested variants show no consistent advantage in token accuracy, predictive loss or exact solving, which remains near zero for all configurations. The results therefore leave the hypothesis open. We specify a confirmatory comparison, with an informative baseline, multiple seeds, controls that isolate nesting and predefined criteria, to determine whether this organisation merits further development.

Mon 19 OctMachine Learning
The gist
Some artificial intelligence models try to think better by applying the same reasoning steps repeatedly. This study tested whether adding an extra step of combining these repeated updates helps these models improve their predictions. The researchers compared standard models with these ‘nested’ versions on a specific benchmark but found no consistent advantage in accuracy or solving ability. The results do not prove the approach is better or worse, leaving the question open for future studies.
Open → 2610.26666v1

Guards safety evaluations sensitive to shortcuts in ai agent actions

Shortcut Sensitivity in Agent-Safety Evaluations

Abstract: AI agents take actions through tools. Some actions are sensitive, so guards that supervise them must themselves be evaluated reliably. Yet the data used to evaluate a guard can include cues that make labels predictable without measuring the intended safety construct, and those cues can also influence guard decisions. We introduce a three-level shortcut-auditing framework and audit completed LinuxArena trajectories and two TS-Bench-derived step datasets. We distinguish label predictability, observational guard association, and paired decision sensitivity. In LinuxArena, shorter average shell commands strongly predict trajectory labels. In AgentDojo-Traj, exact-tool lookup falls from F_1=.901 with mixed-domain folds to .495 under domain holdout. In AgentHarm-Traj, released TS-Guard F_1 falls from .902 under row weighting to .719 with equal interaction weighting, while balanced accuracy rises. We then test the strongest safe-row marker association in a seven-configuration screen on a frozen 64-step AgentDojo-Traj cohort using repeated, interleaved Haiku calls. Rewriting the <INFORMATION> marker lowers blocking by 8.33 percentage points, while opaque tool aliases raise it by 11.46 points. All unsafe calls remain blocked; the changes concern false positives against inherited labels. Both effects remain nonzero after subtracting the formatting-placebo effect, with exploratory 95% ranges excluding zero. The same edits also change measured accuracy against inherited labels. This provides direct evidence that benchmark representation can materially change a guard's decisions and measured evaluation performance. The Haiku-sized effects do not reproduce in Luna or deterministic Q8 TS-Guard. We recommend domain group holdouts, evaluation-unit metrics, and placebo-controlled paired interventions.

Thu 15 OctArtificial IntelligenceCryptography and SecurityMachine Learning
The gist
AI systems use tools to take actions, and some actions need safety checks called guards. The authors found that sometimes these guards rely on shortcuts—simple clues in evaluation data—that don’t really measure true safety. These shortcuts can change how guards decide whether an action is safe, leading to misleading evaluation results. They tested changes to data labels and tools and showed that small representation tweaks can affect guard decisions without changing real safety.
Open → 2610.22243v1

Culture language and region annotations enhance web data benchmarking

FineWeb-CLaR: Culture, Language, and Region Annotations for Benchmark-Aligned Corpus Auditing

Abstract: Cultural evaluation coverage and robustness in language models are difficult to diagnose because pretraining corpora and cultural benchmarks are rarely indexed with comparable metadata. Benchmarks increasingly target culturally situated phenomena at the level of languages, regions, and locale-specific practices, while web-scale corpora are usually organized only by language. A shared culture-language-region layer makes these resources comparable, enabling audits of whether a target cultural phenomenon is represented in pretraining data, evaluated by benchmarks or both. To this end, we introduce FineWeb-CLaR, a large-scale annotated dataset derived from FineWeb and FineWeb-2 that places web documents on a shared culture-language-region axis for corpus auditing and benchmark alignment. FineWeb-CLaR annotates the full 30.9B-document collection from FineWeb and FineWeb-2 with URL-derived region labels and cultural-topic provenance. Our region resolver assigns a non-empty region to 25.61% of documents (7.92B). For cultural-topic analysis, we induce locale-specific topics and project them onto the 14 leaves of the Cultural Taxonomy of Liu et al. (2025), producing Locale Topic Distributions (LTDs) for corpus-side comparison. We also annotate 277 cultural NLP benchmarks with the same taxonomy, language coverage, and region coverage. Together, these resources enable direct comparison between corpus-side pretraining evidence and benchmark-side evaluation coverage.

Mon 21 SeptComputation and Language
The gist
It’s hard to tell if language AI models understand different cultures because the training data and tests are labeled differently. The authors created FineWeb-CLaR, a big dataset that tags web documents with culture, language, and region information, making it easier to compare what’s in the training data with what tests measure. They also labeled many benchmarks to match this system, so it’s clearer if models truly cover cultural topics. This helps check if AI learns from or is tested on diverse cultural content.
Open → 2609.25298v1

Policy ambiguity causes unreliable agent evaluation scores

Policy Loopholes in Agent Evaluation: When Policy Ambiguity Masquerades as Agent Error

Abstract: Agent benchmarks evaluate policy compliance but assume each policy determines a unique correct action. Natural-language policies can violate this assumption through silence, ambiguity, or contradiction, admitting multiple defensible readings that a single gold trajectory cannot capture. Auditing two $τ^2$-bench domains, we develop a taxonomy of such policy loopholes and show that affected tasks produce unreliable scores: they lower scores across different models in different ways and make every model less consistent across repeated trials. A cross-domain comparison reveals that exploitability requires both policy ambiguity and tool permissiveness: when policy complexity exceeds what tools can enforce, agents resolve gaps inconsistently and scores become unreliable. Policy specification quality sets the ceiling on evaluation quality. Benchmark developers should audit policies before collecting gold annotations.

Sun 13 SeptComputation and Language
The gist
Agent benchmarks often assume that a set of rules (policies) clearly say what an agent should do in every situation. This paper shows that real policies written in natural language can be vague or contradictory, allowing multiple right answers that benchmarks don’t capture well. As a result, scores given to agents can fluctuate and be misleading. The authors analyze tasks where policy ambiguity mixes with tool limits, causing inconsistent agent behavior and evaluation. They suggest that carefully checking policies before judging agents can improve evaluation reliability.
Open → 2609.14400v1

Protocol choices strongly affect hardware trojan detection accuracy

Protocol effects on feature-based hardware-Trojan detection across Trust-Hub families

Abstract: Trust-Hub reuses host circuits: several files differ mainly in the inserted Trojan. When gates from sibling variants enter both training and test folds, a detector can benefit from host logic it has already seen. We measure that effect instead of proposing another classifier. The corpus contains 49,124 gates from 16 netlists grouped into five host families. We left the parser, 36 gate features, class weighting, model settings, threshold, and family-level aggregation unchanged and altered one choice: the test boundary. The three settings draw test gates from the pooled corpus, withhold a complete netlist, or withhold every variant of one host. The choice matters. Random forest records F1/AP of 0.914/0.978 with pooled gates, 0.636/0.851 with one netlist held out, and 0.460/0.577 with a host family held out. XGBoost falls from 0.946/0.976 to 0.464/0.544 across the same comparison. Logistic regression loses AP, although its fixed-threshold F1 is not monotonic. Each family shows the same pooled-to-family direction. Feature removal, repeated model and simulator seeds, score normalization, parser-related exclusions, and a smaller sample change the size of the gap without reversing it. Aggregation also matters: a gate-weighted average is dominated by the larger ISCAS files, so the headline values give each host family one vote. Bootstrap and jackknife summaries keep the gap positive, but their folds reuse training families. We treat the five family rows as descriptive evidence rather than independent trials. Five host families are too few for a population claim, and the experiment says nothing about transfer to a new cell library or an industrial design. It supports a narrower conclusion: sibling benchmark variants can inflate apparent transfer. Benchmarks with several variants of one host circuit should report family-aware holdouts and all five family results beside pooled scores.

Mon 7 SeptMachine LearningArtificial IntelligenceComputational Engineering, Finance, and Science
The gist
Detecting hidden harmful circuits called hardware Trojans is important for computer chip security. The researchers found that when testing detection methods, including very similar circuits in both training and testing phases makes the detection seem more accurate than it really is. They showed that using stricter testing — where entire related sets of circuits are withheld — leads to much lower detection scores. This means that how test data is chosen can greatly affect evaluations of security tools.
Open → 2609.07199v1