Llm persona simulation improves cold start ad performance predictions
What Matters for LLM Persona Simulation: Signal Form in Cold-Start E-Commerce Advertising
Machine LearningArtificial Intelligence
Summary
Predicting how well ads will do before they launch is hard without past data. The authors studied how large language models can simulate user responses based on simple persona descriptions. They found that using minimal demographic info without noisy details, along with relative rather than exact cues, and specific prompt wording, makes predictions more accurate. Their approach significantly outperforms other non-LLM and LLM methods at estimating ad click-through rates in a real e-commerce setting.
What this means in practice
- •For digital marketing teams: Improve ad campaign performance predictions before launch using minimal persona descriptions and qualitative signals in large language model simulations.
- •For advertising platform developers: Design ad performance estimation tools that incorporate refined prompt phrasing and demographic anchors to boost prediction accuracy for new campaigns.
Tested on one dataset.
Authors
Jiwoo Choi, In Ik Lee, Woo Sung Seo, Minho Lee, Yun Young Choi
Abstract
Estimating advertising performance in cold-start settings is challenging because historical logs are unavailable before launch. Large language models (LLMs) offer a potential alternative by simulating persona-conditioned user responses, but it remains unclear which design choices actually drive LLM-based persona simulation. Using real Google Ads e-commerce campaign data across five demographic segments, we identify three signal-form dimensions that affect simulation quality: (i) a demographic anchor without noisy rich attributes, (ii) a relative qualitative banner-blindness signal rather than quantitative anchors, and (iii) prompt phrasing. On this campaign, our method attains an MAE of 24.9 and a MAPE of 26.1%, a 51.3% lower MAE than the best tested non-LLM baseline and 77.3% lower than an LLM configuration using explicit CTR anchors. Systematic ablations show that minimal demographic descriptions achieve lower mean error than richer persona configurations, while relative qualitative cues substantially outperform the tested quantitative anchors. We additionally compare these configurations with direct-prior and rule-based baselines and with alternative LLM configurations.