Large language models struggle to mimic individual investor behavior accurately
Beyond Behavioral Accuracy: Evaluating Individual-Level Fidelity in LLM Investor Simulation
Artificial IntelligenceHuman-Computer InteractionInformation Retrieval
Summary
It is hard for AI language models to predict what a single investor will do with their trades over time. The authors studied 80 people who simulated stock trading and found that the AI models could not do better than simple guesses based on recent activity. While the models sometimes matched overall trading patterns, they often missed the details of buying and selling choices. People felt the explanations from the models matched general strategies better than specific decisions. This shows a big difference between matching general actions and truly copying an individual’s logic.
What this means in practice
- •For financial software developers: Improve investor simulation tools by considering limits of current AI models to replicate individual trading decisions.
- •For game designers: Develop more realistic player behavior models by distinguishing between general actions and detailed decision logic.
Authors
Jiajie He, Yuanhong Jiang, Xintong Chen, Dongling Ni, Wenjin Liu
Abstract
Large language models (LLMs) are increasingly used as user simulators, but can they faithfully reproduce an individual's behavior and decision logic over time? We investigate this question in a longitudinal paper-trading study with 80 participants. Using rolling next-day predictions, we evaluate trading activity, buy-sell structure, asset selection, and portfolio outcomes, complemented by controlled input ablations and participant-rated explanation alignment. No evaluated LLM reliably outperforms a simple recent-activity persistence baseline for predicting trade occurrence, while buy-sell structure and asset selection remain difficult to reproduce. Similar activity-level performance can also yield substantially different portfolio trajectories. Ablations show that activity predictions depend primarily on recent trades, whereas asset choices are sensitive to multiple evidence sources. Participants rate explanations as more aligned with their recurring strategies than with their decision-specific rationales. These findings reveal a gap between coarse behavioral agreement and individual-level fidelity, motivating separate evaluation of behavioral accuracy, explanation alignment, and evidence dependence in personalized user simulation.