Papers for

qa system developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Uncertainty signals hallucination risk when retrieval supports answers

When Does Uncertainty Measure RAG Hallucination Risk?

Abstract: Uncertainty can measure hallucination risk in retrieval-augmented generation (RAG), but only when retrieved evidence constrains the answer. When evidence is present, unsupported factual claims become less likely; when retrieval coverage is poor, the model can answer from internal knowledge and assign high probability to unsupported claims. We test this across four instruction-tuned models on HotpotQA and SQuAD. After matching faithful and hallucinated answers within word-count bins, length-only classifiers fall near chance, while perplexity reaches 0.81--0.91 AUROC on targeted unsupported edits. Varying retrieval depth shows why the score changes meaning: detection can weaken with only the top-ranked passage, then recover after adding a second passage. A combined uncertainty and semantic-drift score improves calibration, and measurements remain above chance on RAGTruth QA. Results support a conditional view of uncertainty as a useful measure of RAG hallucination risk.

Thu 15 OctComputation and Language
The gist
Some AI models generate answers using retrieved documents to avoid making things up, called hallucinations. The authors show that uncertainty measures can spot hallucinations well only when the retrieved information actually limits what the AI can say. They tested this by comparing AI answers with and without supporting evidence and found uncertainty aligned better with hallucinations when more retrieved data was used. Combining uncertainty with a semantic drift score made detecting hallucinations even more reliable.
Open → 2610.22615v1

Hallucination detection labels differ based on faithfulness versus factual correctness

The Labeling Problem in Hallucination Detection Benchmarks: An Empirical Evaluation

Abstract: In recent years, several methods for detecting when large language models (LLMs) hallucinate have been developed. These methods are often benchmarked with open-domain question answering (QA) datasets containing questions and corresponding short reference answers. First, an LLM is used to generate answers to questions within the QA dataset. Then, some automated labeling strategy is used to label these answers as hallucinated or not by comparing them with the reference answers in the dataset. This evaluation setting creates a methodological ambiguity between two criteria: reference faithfulness (whether the answer is fully supported by the reference) and factual correctness (whether the answer is free from contradictions and factually false specific claims). In practice, automated labelers may apply the former criterion even when the intended target is the latter. We study this potential criterion mismatch using 900 human-labeled question-answer pairs spanning three commonly used QA datasets and three generator models, with labels targeting answer-level factual correctness. We evaluate lexical similarity metrics, a reference-entailment NLI baseline, and seven LLM judges under controlled prompt variants as automated labelers. Our experiments reveal substantial disagreement both among automated labeling strategies and between these labels and human annotations. Many strategies also exhibit strong directional error biases, and for most judge-generator pairs, replacing a faithfulness-oriented prompt with a factual-correctness prompt improves agreement with human annotations and reduces false-positive dominance, indicating that automated hallucination labels depend strongly on how the target criterion is specified. Label-source choice should therefore be considered a fundamental part of benchmark design and made explicit, validated, and matched with the benchmark goal.

Tue 6 OctComputation and Language
The gist
Figuring out when AI language models make things up—called hallucinations—is tricky because there are different ways to judge if an answer is wrong. One way is to check if the answer matches a given reference, and another is to see if the answer is factually true regardless of that reference. The authors found that automatic systems often get confused between these two and label answers inconsistently compared to human judgment. They showed that changing how the question is asked to AI can improve agreement with humans. This means careful design and clarity about what kind of error is being labeled are very important.
Open → 2610.08026v1

Selective inference improves tree structured decision making in reinforcement learning

SIPO: Selective-Inference Policy Optimization for Tree-Structured Agentic RL

Abstract: Tree-structured reinforcement learning trains search agents by comparing alternative continuations and propagating terminal rewards to intermediate decisions. Adaptive expansion, however, creates a statistical asymmetry: an incumbent is selected using its own generation statistic, whereas fresh siblings are sampled after selection. When that statistic is associated with return, branch values can reflect selection history as well as continuation quality, even for a shared parent. We propose Selective-Inference Policy Optimization (\SIPO{}), which incorporates this distinction into tree-based credit estimation. Its scale-free branch criterion keeps generation scores and sibling penalties on a consistent relative scale; exchangeable branching supplies multiple fresh continuations from each selected parent; and order-statistic correction adjusts retained incumbent values using selection rank and the estimated score--outcome association. These mechanisms preserve the leaf budget and the host policy optimisation objective. Across seven QA benchmarks using Qwen3-4B, Qwen3-8B, and Qwen2.5-7B, \SIPO{} achieves the highest reported multi-hop and single-hop averages among the compared methods. On Qwen3-8B, it improves these averages over AT\textsuperscript{2}PO by $1.31$ and $1.07$ percentage points, respectively, and ranks first on six of seven benchmarks. Component ablations evaluate the individual and combined changes, while early-training paired diagnostics show a selected--fresh value gap alongside a near-zero fresh--fresh reference. Together, these results support accounting for selection history when constructing and evaluating search-agent rollouts. Our code is available at https://github.com/Zenghuang-Fu/SIPO

Mon 28 SeptArtificial Intelligence
The gist
When computers try to make decisions by exploring different possible future moves, they often pick choices based on past results. But this can cause a problem because some options look better just because they were selected before, not necessarily because they truly are better. The authors found a way to correct for this bias by carefully re-evaluating choices and their alternatives, making the decision-making process fairer and more accurate. Their method, called SIPO, showed better performance in question-answering tests using large language models.
Open → 2609.34805v1

Knowledge graph method measures large language model context understanding

Do LLMs Understand Context? A Knowledge Graph-Based Evaluation Framework

Abstract: While large language models (LLMs) have achieved remarkable linguistic capabilities, a profound question lingers at their core: do these models truly comprehend context or simply excel at pattern matching on an unprecedented scale? Contextual understanding in LLMs refers to the ability to correctly extract relevant information from a given context, integrate it into a coherent internal representation, and reason over it to produce factually consistent and contextually grounded responses. However, traditional methods such as BiLingual Evaluation Understudy (BLEU) and perplexity simply measure surface-level performance. This reveals a critical gap in question answering (QA), where responses must be contextually grounded rather than simply being memorized associations. To fill this void, we propose a novel knowledge graph (KG) based evaluation framework for LLM contextual understanding in QA. Central to this is Semantic Structural Similarity for KGs (S3KG), a hybrid similarity measure combining structural and semantic signals into a single score. In addition, a diagnostic analysis framework is developed to identify and categorize reasoning errors at the triplet level, enabling fine-grained analysis of model failures. Together, across nine benchmarks, S3KG achieves F1 gains of up to $+7.6$ points over the strongest baseline and AUROC up to $0.973$.

Thu 24 SeptArtificial IntelligenceMachine Learning
The gist
Large language models can produce impressive answers but it’s unclear if they truly understand the information around them or just guess based on patterns. The authors developed a way to check if these models really get the context by comparing their answers to knowledge graphs, which show facts and their connections. They created a special score to see how well the model's answers match the structure and meaning of these facts. This method also helps spot exactly where models make reasoning mistakes. Their tests on multiple benchmarks showed better results than previous tools.
Open → 2609.30484v1

Confidence signals in language models reveal instability in answers

When Confidence Signals Disagree: Local and Global Confidence in Autoregressive Language Models

Abstract: Modern predictive systems expose multiple quantities that are commonly interpreted as measures of confidence. However, these quantities can summarize different aspects of the predictive process. This distinction matters when confidence is used to evaluate reliability or inform downstream oversight and control. We investigate whether different confidence readouts are empirically interchangeable in an autoregressive language model by comparing local confidence, defined from the probability of the greedy-selected answer token, with global confidence, defined from modal-answer frequency under repeated sampling. Across MMLU and ARC Challenge, the two signals are weakly correlated and differ substantially in their association with correctness: global confidence is moderately associated with correctness, whereas local confidence shows little association. We further test whether question-level disagreement between the signals is associated with sampling instability. On ARC, larger local--global confidence gaps are associated with higher answer entropy, more distinct sampled answers, and lower modal-answer concentration. The gap--entropy association persists when disagreement and instability are estimated from disjoint stochastic samples, indicating that it is not explained by shared finite-sample variation. The corresponding relationship is substantially weaker on MMLU, where only 4% of questions exhibit sampling instability. These results show that confidence readouts derived from the same predictive system are not empirically interchangeable and that their disagreement can provide a diagnostic of unstable sampling behavior. Confidence should therefore be treated as an explicitly defined measurement rather than as a single intrinsic scalar property of a model, particularly when it is used to inform downstream evaluation, oversight, or control.

Tue 15 SeptMachine LearningArtificial Intelligence
The gist
This paper looks at two ways language models show how sure they are about their answers. One way checks how likely a model thinks its immediate answer token is, and the other counts how often the same answer shows up in many tries. The researchers found these two signals do not always agree and that the global confidence (the repeated-answer count) is better linked to correct answers. When the two signals differ a lot, it often means the model's answers are unstable or uncertain, especially on harder questions. This shows confidence is not one simple number but several measurements that tell different stories.
Open → 2609.16933v1