Large language models improved by using explicit logical structure
When Form Changes but Logic Doesn't: Building Logic-invariant LLMs through Logical Structure
Artificial Intelligence
Summary
Large language models can perform reasoning tasks but often struggle to give consistent answers to questions that are logically the same but worded differently. The authors show that current models do not always recognize such logical equivalences and fail to generalize to new logical forms. They created a method called LoGIST that helps models understand the underlying logic by turning inputs into structured diagrams and using those to guide the models' reasoning. This approach makes models more consistent and better at handling new logical expressions.
What this means in practice
- •For natural language processing engineers: Improve language model consistency in reasoning tasks by integrating structured logical guides into input processing pipelines.
- •For software developers building ai assistants: Enhance AI assistants to provide logically consistent answers despite variations in question phrasing by embedding logical structure awareness.
Authors
Xuyuan Liu, Xinshuai Dong, Elynn Chen, Yujun Yan
Abstract
Large language models (LLMs) have achieved strong performance across diverse reasoning tasks, yet it remains unclear whether this performance reflects an understanding of underlying logical structure or a reliance on shallow semantic cues. A key indicator of such understanding is whether models treat logically equivalent inputs consistently, producing the same predictions despite differences in expression. Through systematic evaluation on curated data, we show that even strong LLMs often fail this test: they do not consistently preserve predictions across logically equivalent inputs and generalize poorly to unseen logical forms. To address this limitation, we propose LoGIcal STructure-guided reasoning (LoGIST), a framework that guides LLM reasoning with explicit logical structure. LoGIST maps each input instance to a decision diagram, a directed acyclic graph that captures the underlying logical structure, and encodes this graph into a logical embedding that conditions the LLM's reasoning process. Across multiple training settings, LoGIST reduces inconsistency under logical equivalence and improves generalization to unseen logical forms, suggesting that logical invariance is a useful target for building more robust and generalizable logical reasoning in LLMs.