Large language models guided to follow letter or spirit of law
Directing large language models to follow the letter or spirit of the law
Computation and Language
Summary
Deciding whether to follow the exact words of a rule or its intended meaning is a common challenge. The authors developed a way to adjust large language models so they can focus on either the letter or the spirit of legal rules. Their method changes how the models behave across different scenarios and legal cases with only small tweaks. They also found patterns inside the models that match formal ideas about legal concepts, helping us understand and control machine legal reasoning better.
What this means in practice
- •For legal technology developers: Create AI tools that can be tuned to interpret legal rules strictly or flexibly in automated document review and advisory systems.$Commercial implications: Enables development of customizable legal AI products that better align with client preferences for rule interpretation.
- •For compliance teams: Use adjusted language models to assess whether actions comply with laws based on literal text or intended purpose in varied regulatory environments.
Authors
Peng Qian, Andrew Li, Sam Chen, Sonia K. Murthy, Yonatan Belinkov, Tomer D. Ullman
Abstract
The distinction between the spirit and letter of the law is a central issue across research and everyday life, and a growing concern for building safe, intelligent machines. What is this distinction based on, and how can we develop machines that follow the intention behind a rule? We used targeted adaptation that made large language models prioritize the spirit or letter of the law. With minimal modifications, our method significantly changed LLM behavior across diverse measures, novel vignettes, real-world scenarios, and influential legal cases. An analysis of model internals revealed a low-dimensional space with three interpretable dimensions matching a formal pre-specified framework for the geometry of legal concepts. These findings show how legal thought in LLMs may be organized and directed.