Language models show consistent beliefs that predict responses
Beliefs and Behavior in Language Models
Artificial IntelligenceMachine Learning
Summary
It is unclear whether big AI language models really have something like beliefs or desires inside them, which people often mention to explain their behavior. The authors study whether we can treat certain hidden variables, drawn from the models' answers, as a kind of belief that helps predict what the models will say next. They found that more capable models behave in ways consistent with holding such beliefs, making it easier to predict their answers. They also explore how these beliefs change when the model processes reasoning tasks and how well models follow instructions based on these beliefs.
What this means in practice
- •For ai safety teams: Assess language model behaviors by inferring internal beliefs to better predict harmful or undesirable outputs.
- •For chatbot developers: Use inferred belief states from language models to improve chatbot responses and adherence to instructions during conversations.
Authors
Alex Smolin, Bryan Wilder
Abstract
There is significant uncertainty about whether abstractions like beliefs or desires usefully describe the behavior of large language models (LLMs). In addition to the inherent scientific interest of this question, these latent quantities are often invoked to explain the behavior of LLMs to users or to define and evaluate harmful behaviors which are relative to intent. Nevertheless, we currently lack a means to systematically test whether concepts like "belief" are well-applied to LLMs, and hence whether they are likely to be fruitful ingredients of attempts to align models with human interests. We propose an approach for empirically studying such questions, asking whether a single latent variable inferred from the LLMs' outputs -- interpreted as a degree of belief -- allows an observer to make interpretable predictions of how the LLMs' will respond to new prompts. We find that highly capable models are usefully described as holding beliefs and that, generally, the predictability of model outputs based on an inferred latent belief tracks overall trends in model capability. Building on these findings, we provide empirical strategies to study how beliefs in LLMs can be measured, the extent to which LLMs comply with instructed decision rules or payoffs, and how beliefs evolve within individual instances of an LLM over the course of reasoning.