Analysis clarifies when linear representation claims are meaningful
A Survey on the Linear Representation Hypothesis
Artificial Intelligence
Summary
There is a popular idea called the linear representation hypothesis (LRH) that tries to explain how information can be stored in AI models and brains. However, the authors find that past work often treats this idea inconsistently, making it hard to test or disprove. They suggest a clearer way to define the LRH by specifying the model details, where the representation is, what features mean, and the dataset used. This helps researchers make clear claims that can be tested scientifically. The paper also points out important unanswered questions to guide future research.
What this means in practice
- •For machine learning engineers: Design clearer evaluation protocols for interpreting neural network internal features based on model, feature, and dataset specifics.
- •For neuroscience data analysts: Use the clarified hypothesis framework to better relate brain imaging patterns to hypothesized linear representations.
A survey. It maps existing work.
Authors
Sewoong Lee, Marc E. Canby, Ikhyun Cho, Julia Hockenmaier
Abstract
The term "linear representation hypothesis" (LRH) has appeared across diverse subfields of artificial intelligence, neuroscience, and cognitive science. But previous works have not consistently treated the LRH as a falsifiable scientific hypothesis; we analyze these inconsistencies and examine their implications for how prior theoretical and methodological results should be interpreted. Based on this analysis, we argue that claims regarding linear representations become well-defined only through careful examination of the model, representation location, feature definition, and evaluation dataset. We therefore propose a more rigorous formalization of the LRH that makes these dependencies explicit and allows the hypothesis to be evaluated as a falsifiable scientific claim. Finally, we identify some non-trivial open problems that warrant further attention from the research community.