Large language models supply partial lexical knowledge for logic proofs
Explaining Textual Entailment with Lexical Entailments: Using LLMs to Supply Lexical Relations for Formal Proofs
Computation and LanguageArtificial Intelligence
Summary
Natural language reasoning often needs facts about words and their meanings, but it is unclear if large language models (LLMs) use such facts correctly during reasoning. The authors studied if LLMs can provide word-level knowledge to help formal logic systems prove sentence meanings from other sentences. They created a new dataset to test if models can explain sentence relationships by listing word relationships, then tried using those word relations in a logic prover. The study found that even powerful LLMs struggle with this, and their word relations are sometimes only partly correct or too tailored to specific sentence pairs.
What this means in practice
- •For natural language processing engineers: Use LLM-generated lexical relations to improve reasoning in logic-based NLI systems when formal proofs need word-level knowledge.
- •For ai system integrators: Augment existing neuro-symbolic systems by incorporating partial lexical knowledge from LLMs to assist transparent natural language understanding.
Authors
Jorryt de Jong, Stefan Moraca, Ettore Cesari, Lasha Abzianidze
Abstract
Large Language Models (LLMs) are highly capable of natural language reasoning and appear to store a great deal of lexical knowledge, but it is still unclear how much of this knowledge they actually use when reasoning, and whether they use it in the right way. On the other hand, logic-based Natural Language Inference (NLI) systems provide transparent and formally grounded reasoning, but they need to be supplied with rich lexical knowledge to prove inferences beyond purely logical ones. In this paper, we evaluate whether LLMs can identify all lexical knowledge needed to solve NLI problems and how much this knowledge contributes to proof search in a logic-based NLI system. Our research focuses exclusively on structured lexical entailments (e.g., chinchilla$\sqsubseteq$small animal) as a proxy for structured explanations for NLI problems with an entailment label. First, we curate a dataset for a new task of explaining sentential entailments with a set of lexical entailments. The dataset is used to intrinsically evaluate LLMs on generating structured lexical explanations. Then, we use NLI as an extrinsic evaluation in a simple neuro-symbolic setting, assessing whether LLMs can supply sufficient lexical relations to LangPro, a natural-logic theorem prover for natural language. The results show that the proposed task remains challenging even for hosted proprietary LLMs, and that their contribution to theorem proving is moderate: generated relations are often only partially sound and may be tailored to the specific NLI problem rather than representing generally valid lexical knowledge.