Legal factors are identifiable inside large language models

Where Do Legal Factors Live in LLMs?

Computation and Language

Summary

In legal reasoning, experts look at important facts from past cases to decide new cases. The authors checked whether big AI language models recognize these important legal fact patterns inside themselves. They found that all 26 expert-defined legal factors can be found in the AI models’ inner workings. This means the models hold knowledge similar to legal concepts experts use, making it possible to connect AI understanding and symbolic legal ideas.

What this means in practice

  • For legal technology developers: Identify and extract expert-defined legal factors directly from AI model activations to improve explainability of AI-driven legal tools.
  • For compliance teams: Audit and verify AI decisions in legal contexts by linking model activations to known legal factors for transparency.

Authors

Li Zhang, Kevin Ashley

Abstract

In case-based legal reasoning, legal professionals identify legally significant fact patterns in a new dispute and compare them with those of decided cases. In AI & Law, HYPO dimensions and CATO factors represent these patterns symbolically, organizing expert knowledge distilled from case narratives as explicit, structured, interpretable, and auditable legal concepts and relations that support case comparison, legal argument, and precedent-based reasoning. Large language models (LLMs) are increasingly used for legal tasks, yet their internal organization of legal knowledge remains difficult to trace. We investigate whether LLMs form internal representations corresponding to all 26 CATO factors. Two research questions examine linear decodability and sparse localization with sparse autoencoders. Experiments cover three open models, Qwen3-1.7B, Qwen3-8B, and Gemma-2 2B, supported by cross-model comparison and human audit of controlled natural-language passages. Results show that all 26 factors are linearly decodable in all three models. Single-feature sparse readouts cover the complete factor catalog, with best-layer test AUCs ranging from 0.86 to 1.00. Five selected features approach full-vector readout performance. We report 78 model-specific layer-feature coordinates, one for each factor in each model. Together, these results show that information corresponding to expert-defined legal factors can be read from the internal activations of LLMs processing legal narratives. By locating these readouts, our study connects legal language and the symbolic abstractions of HYPO and CATO to learned internal representations.