Reasoning graph reliability varies with model and impacts answer confidence

Diagnostic Analysis for Large Language Model Reasoning

Computation and Language

Summary

Figuring out how reliable large language models are when they create reasoning chains or graphs is important. The authors test these reasoning structures in two steps using complex questions. They find the reliability changes based on how the graphs are built, the model used, and how prompts are designed. Using uncertain graphs reduces performance, but when perfect graphs are used, having the graph helps in telling right answers from wrong ones by looking at confidence scores.

What this means in practice

  • For ai product teams: Improve AI question-answering products by assessing and enhancing reasoning graph reliability to boost answer confidence and error detection.
  • For data validation teams: Use graph-based diagnostics to better detect unreliable reasoning paths and improve trustworthiness of automated data analyses.

Authors

Xinyu Liu, Dongqi Fu, Jun Wu

Abstract

Reasoning structures (e.g., chains and graphs) are increasingly used for probabilistic inference, factuality verification, and uncertainty estimation, but their structural reliability is rarely evaluated directly. We test whether this reliability holds through a proposed two-stage diagnostic evaluation in open-ended multi-hop QA. Stage 1 tests different capabilities required to construct reasoning graphs, showing that graph reliability varies with topology, prompt design, and model family. Stage 2 uses uncertainty estimation as a downstream test case. Generated-edge experiments show that graph-based computation is sensitive to upstream graph errors, with performance dropping relative to the gold-graph setting. Gold-graph experiments further show that graph structure provides little benefit when used only as prompt context, but improves the separation of correct from incorrect answers by confidence score and enables node-level diagnostics.