Papers for

ai testing teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Generative model evaluation improved by sparse and unpaired query matching

Benchmark Prediction with Sparse and Unpaired Cached Responses

Abstract: Evaluating a generative model on all queries from all available benchmark tasks is prohibitively expensive. As such, models are often evaluated on different subsets of tasks and/or different subsets of queries from a given task. The resulting collection of responses and the corresponding (model, task) score matrix are sparse and unpaired. In this paper, we introduce the Product Kernel Perspective Space (PKPS), a collection of low-dimensional representations of models based on their embedded responses. PKPS generalizes existing methods for representing black-box generative models by comparing responses to similar -- rather than just identical -- queries and is therefore able to utilize sparse and unpaired information. Theoretically, we show that benchmark prediction using PKPS is query-efficient relative to methods that only use information from dense and paired portions of the evaluation data. Empirically, we show that PKPS-based methods outperform baselines on benchmark prediction across two large benchmark suites.

Mon 2 NovMachine Learning
The gist
Testing AI models on every question from every test is very slow and expensive. The authors introduce a new way called Product Kernel Perspective Space (PKPS) to compare AI model answers even when tests only cover some questions for some models. This method uses similarities between different questions, not just identical ones, making it faster and requiring fewer examples. Their approach shows better results when predicting how AI models will perform across many tasks.
Open → 2611.03614v1

Multi-image model failures traced to identifying image instances

What Breaks in Multi-Image Schema Application? Factorial Oracle Interventions Reveal an Instance-Information Bottleneck

Abstract: Modern vision-language models can accept multi-image contexts, but aggregate accuracy rarely reveals where a multi-image reasoning pipeline fails. We study a controlled schema-application task in which a model must locate two target characters in a 16-image sequence, identify their visual attributes, consult an item-specific visual codebook, and compose two ordered nonce code names. We introduce a within-item 2-by-2 factorial Oracle design: Oracle-A supplies a verified textual schema, Oracle-B supplies a verified ordered target-appearance representation, and both interventions retain the original images and final question. Across 72 new items and 1,188 analyzed model responses, Oracle-B increased strict exact accuracy by 54.6 percentage points (95% confidence interval: 46.3 to 63.0) and outperformed Oracle-A by 60.6 points (Holm-adjusted p-value no greater than .0002). An auxiliary retrieval diagnostic reached only 49.1% ordered-appearance accuracy even after removing code-name generation. Providing both Oracle sources yielded 100% accuracy across all models. These results localize the dominant recoverable deficit to upstream ordered instance recovery rather than primarily to downstream schema use and composition.

Wed 14 OctComputer Vision and Pattern Recognition
The gist
Some vision-language AI models look at many images and answer questions about them, but it can be hard to know where they go wrong. The authors studied a task where a model had to find two characters in a series of pictures and name them based on their looks. They tested the model with helpers that gave it the exact character information or the exact character descriptions. The biggest improvement came from giving the model the exact descriptions of the characters’ appearances, showing that the main problem is recognizing and ordering those characters before using the rules to name them.
Open → 2610.19611v1

Parametric memory influences large language models reasoning performance

MemoReason: Evaluating the Effect of Parametric Memory on Contextual Reasoning in LLMs

Abstract: Large Language Models (LLMs) perform well on reasoning benchmarks, but it remains unclear whether this reflects genuine contextual reasoning or reliance on facts memorized in their parameters. We investigate this by distinguishing two possibilities: a broad \textit{memorization bias}, where familiar content improves reasoning performance, and the \textit{Strong Parametric Shortcut Hypothesis}, where models skip reasoning entirely and recall stored answers. To test these effects, we introduce \textbf{MemoReason}, a human-curated benchmark that pairs factual reasoning tasks with structurally identical \fictitiousterm{} versions where real entities like people, companies, or dates are systematically replaced by \fictitiousterm{} ones of the same type. This \scorerevision{preserves task structure and specified reasoning operations} while varying the familiarity of the context, allowing controlled measurement of how the parametric memory affects reasoning. \revision{Our evaluation of recent LLMs reveals consistent and statistically significant performance drops of up to 15.7\% in the fictitious setting, demonstrating a clear memorization bias.} However, a targeted analysis of \revision{questions failed in the fictitious setting} shows that models rarely respond with the corresponding factual answer, indicating that direct parametric shortcuts are not the dominant failure mode. These findings suggest that parametric memory influences reasoning through mechanisms more complex than simple factual recall. \textbf{MemoReason} provides a controlled framework for studying these mechanisms and for extending paired factual-fictitious{} evaluation to broader reasoning settings.

Mon 28 SeptComputation and Language
The gist
Large language models (LLMs) do well on reasoning tests, but it was unclear if they actually think through problems or just remember facts they learned before. The authors created a test called MemoReason, which swaps real facts for made-up ones while keeping the questions the same. They found that the models perform worse with made-up facts, showing they rely on memory, but they don’t just spit out memorized answers without reasoning. This means the models’ memory affects reasoning in more complex ways than just recalling facts.
Open → 2609.35312v1

Cosine similarity can miss key dialogue model structures despite linear access

When Cosine Similarity Fails to Reflect Linearly Accessible Structure in Dialogue Models

Abstract: Cosine similarity is widely used to analyze transformer representations, implicitly assuming that similarity reflects task-relevant structure. We study when this assumption fails in dialogue-conditioned large language models. Across three 7-8B chat-tuned models, ambient cosine similarity substantially underestimates linearly decodable persona structure on the same hidden states; numerically, linear probe AUC is in the 0.73-0.97 range while cosine kNN is in the 0.56-0.77 range on a 30-class task. A low-dimensional supervised subspace recovers much of this gap, whereas a matched-rank PCA subspace does not and in some cases degrades performance. This mismatch is regime-dependent: it is absent in single-sentence sentiment classification (SST-5), and a matched-cardinality control rules out attribute cardinality as a confound. The gap does not systematically increase across dialogue turns, and the task-aligned subspace remains stable over time. However, two of three models violate a pre-registered within-subspace separability invariance criterion (|Delta AUC| <= 0.03), and one model violates a pre-registered turn-invariance criterion (|Delta L| <= 0.05). These results show that cosine similarity can fail to reflect task-aligned structure in dialogue representations even when that structure is linearly accessible.

Fri 18 SeptComputation and Language
The gist
Cosine similarity is a common way to measure how alike two sets of words or meanings are inside language models. The authors show that in chat-focused language models, cosine similarity often doesn’t capture important personality-related patterns that are actually easy to find with other methods. They find that some ways to focus on important features recover these patterns, but simple dimension reduction like PCA doesn’t help. This problem doesn’t happen in simple tasks like sentiment analysis, and it doesn’t get worse over the course of a conversation.
Open → 2609.22522v1