Large language models show knowledge access depends on query geometry
The Geometry of Knowledge Accessibility in Large Language Models
Computation and Language
Summary
Large language models hold lots of information but don’t always find what you ask for reliably. The authors discovered that the way a question is represented inside the model forms a shape where easier questions are near a center point and harder ones are farther away. This shape helps predict how well the model can access knowledge before it starts answering. Different strategies improve answers based on where a question lies in this space, offering ways to make smarter use of models.
What this means in practice
- •For ai application developers: Adapt inference methods based on a query’s position in the model’s representation space to improve answer reliability.
- •For customer support teams: Use query geometry to determine when to apply query rewriting, chain-of-thought prompts, or retrieval tools to improve answer quality.
Authors
Lihu Chen
Abstract
Large language models (LLMs) contain broad knowledge, but they cannot access all of it reliably. We study this problem through knowledge accessibility, which describes whether the knowledge needed for a query can be recalled from the model. We find that knowledge accessibility has a simple geometric structure in the model's representation of the query alone, before any generation. More accessible queries are closer to a center in the representation space, while less accessible queries are farther away. This geometry reveals a knowledge boundary that separates more accessible queries from less accessible ones. Accessibility consistently decreases with distance from the center, and this distance-based ordering transfers across datasets even when the centers differ. Controlled experiments further show that the centered geometry is more closely related to knowledge accessibility than to reasoning difficulty. The geometry also reveals when different interventions are useful. Query rewriting helps more for accessible queries, chain-of-thought reasoning helps more near the boundary, and retrieval gives larger gains beyond the boundary. These findings not only provide a new geometric view of how knowledge is organized in language models, but also suggest a useful pre-generation signal for adaptive inference.