Graph text formats affect reasoning accuracy in large language models
Graph Serializations for Reasoning LLMs: A Study Across Knowledge Extraction and Structural Reasoning
Artificial Intelligence
Summary
How knowledge graphs are turned into text for large language models (LLMs) can make a big difference in how well the models understand and reason about the information. The authors studied 15 different text formats and tested them with nine advanced LLMs on tasks involving extracting knowledge and reasoning about graph structure. They found that while extracting facts is mostly unaffected by the format, the ability to solve reasoning problems depends heavily on how the graph is written out. They suggest choosing graph text formats dynamically based on the specific model and task to get the best results.
What this means in practice
- •For knowledge engineers: Select the best graph text format per model and task to improve automated ontology knowledge extraction accuracy.
- •For ai product developers: Optimize graph serialization choices dynamically within LLM-based reasoning tools to enhance structural query accuracy in AI applications.$Commercial implications: Enables building more accurate AI services for graph-based reasoning by tailoring input formats, improving competitive edge in intelligent assistants.
Authors
Julien Yu, Mihaela Bornea, Avirup Sil, Katrin Kirchhoff
Abstract
The way a knowledge graph is serialized as text for inclusion into LLM context can substantially affect downstream LLM performance, with prior work reporting tens of percentage points variations across formats. However, existing studies consider only a limited set of representations and predate frontier reasoning models, which now offer stronger reasoning, instruction-following, and long-context capabilities. We evaluate 15 graph representation formats across nine frontier LLM configurations on ontology-based knowledge extraction and structural graph reasoning tasks. While knowledge extraction is relatively insensitive to different serialization formats, the choice of the optimal format for structural reasoning tasks significantly affects accuracy. We demonstrate that the optimal representation choice depends on model, query, and task and propose to approach graph serialization as a dynamic orchestration decision.