Summary
Simulations using large language models (LLMs) often check if people act the same way as the real people they mimic, but this only shows what people do, not why. The authors argue that understanding why people act a certain way is important for better social simulations. They introduce a new idea called representational adequacy, which checks if the simulated reasoning behind actions matches real human thinking. This helps simulations explain behavior, diagnose problems, and test interventions more accurately. The paper also clarifies how this idea is different from other evaluation methods and highlights challenges in measuring it.
What this means in practice
- •For social simulation developers: Design simulations that better capture why people behave certain ways, improving scenario realism and interpretability.
- •For policy analysts: Assess social interventions more effectively by simulating reasoning behind human actions, not just observed behaviors.
A position paper. It proposes an approach and reports no results.
Abstract
LLM-based social simulations are primarily evaluated for behavioral fit, testing whether agents reproduce the actions or response distributions of the people they are simulating. However, the promise of simulation extends beyond behavioral fit. Simulations can explain human behavior, diagnose barriers, and compare large-scale interventions. These use cases depend on understanding \textit{why} people acted a certain way, not just \textit{what} they did. As a result, behavioral fit is insufficient for these types of claims because behavior underdetermines the reasoning process behind it. For instance, the behavior of staying silent may be due to disinterest or suppressed speech, and not answering a call may be due to distrust of the caller or limited phone access. In this paper, we propose \textit{representational adequacy} as a new evaluation target for LLM-based social simulations. By leveraging LLM reasoning traces, representational adequacy measures whether a simulation's scenario--reasoning--action triples preserve the reasoning process behind the behavior in a way that is faithful to the population and scenarios being simulated. We distinguish representational adequacy from interpretability and alignment metrics, propose ways to integrate it into simulation research, and pose its measurement as an open problem.