Language models can be described as holding beliefs that guide answers
Beliefs and Behavior in Language Models
Artificial IntelligenceMachine Learning
Summary
It is unclear if ideas like 'beliefs' about facts really apply to computer language programs called large language models (LLMs). The paper explores a new way to test if these programs act as if they hold beliefs by looking at patterns in their answers. The authors find that for smarter models, describing their behavior as having beliefs helps predict how they respond to new questions. They also show how to study whether models follow rules they are given and how their 'beliefs' can change during reasoning. This helps us better understand how to think about and work with these AI systems.
Large language modelsBeliefsLatent variablesModel capabilityDecision rulesIntentReasoningAlignmentPredictabilityEmpirical methods
Authors
Alex Smolin, Bryan Wilder
Abstract
There is significant uncertainty about whether abstractions like beliefs or desires usefully describe the behavior of large language models (LLMs). In addition to the inherent scientific interest of this question, these latent quantities are often invoked to explain the behavior of LLMs to users or to define and evaluate harmful behaviors which are relative to intent. Nevertheless, we currently lack a means to systematically test whether concepts like "belief" are well-applied to LLMs, and hence whether they are likely to be fruitful ingredients of attempts to align models with human interests. We propose an approach for empirically studying such questions, asking whether a single latent variable inferred from the LLMs' outputs -- interpreted as a degree of belief -- allows an observer to make interpretable predictions of how the LLMs' will respond to new prompts. We find that highly capable models are usefully described as holding beliefs and that, generally, the predictability of model outputs based on an inferred latent belief tracks overall trends in model capability. Building on these findings, we provide empirical strategies to study how beliefs in LLMs can be measured, the extent to which LLMs comply with instructed decision rules or payoffs, and how beliefs evolve within individual instances of an LLM over the course of reasoning.