Papers for

market researchers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

LLM digital twins improve response prediction using structured survey profiles

Structuring Survey Histories for LLM Digital Twins: A Controlled Study of Background, Decision Procedure, and Evaluation Profiles

Abstract: LLM-based digital twins aim to predict how individuals would respond in new settings from representations of their prior responses. Existing evidence that compression of survey histories need not reduce predictive accuracy suggests that representation structure, rather than information volume alone, may be an important design choice. We study BDE, a fixed representation that organizes each respondent's survey history into Background, Decision procedure, and Evaluation. We evaluate it on Twin-2K-500 (Toubia et al., 2025) by predicting each respondent's held-out answers on 17 tasks. In a paired evaluation with 100 respondents, a fully inferred BDE profile improves overall accuracy over the raw survey transcript by 1.27 percentage points (pp) on gpt-5.4-nano (p < 0.01) and 1.19 pp on gpt-5.4-mini (p < 0.001). Three analyses then examine the schema, extraction depth, and component composition. Against a length-matched free-form profile written from the same transcript by the same extractor, BDE improves pricing accuracy on both simulators (2.15 pp on nano, p < 0.01; 1.38 pp on mini, p < 0.05), while its overall advantage is significant only on nano (1.05 pp, p < 0.05). Allowing deeper inference during extraction yields no statistically significant gain on either simulator. By contrast, adding Evaluation to a Background--Decision procedure profile improves overall accuracy by 1.44 pp on nano (p < 0.001) and 1.22 pp on mini (p < 0.001). These results indicate that the advantage over raw transcripts is not explained by compression or length alone, and attribute it primarily to profile composition rather than deeper extraction.

Thu 5 NovComputation and Language
The gist
The paper finds that how survey answers are organized matters for predicting what people will say in new situations using language AI models. Instead of just compressing all survey answers, the authors show that dividing information into three parts—Background, Decision procedure, and Evaluation—helps make better predictions. They tested this idea by comparing these structured profiles to raw survey transcripts and other formats, showing small but consistent improvements. The main advantage comes from the way information is grouped, not from making the data shorter or going deeper in analysis.
Open → 2611.08730v1

Human and AI work together to improve qualitative coding accuracy

Scaling Qualitative Analysis with Large Language Models: Development and Validation of the Smart Coder Framework for Human-LLM Collaborative Coding

Abstract: Current uses of large language models (LLMs) for qualitative coding often rely on ad hoc prompting with limited guidance for reliability and human oversight. To address these limitations, we propose the Smart Coder Framework, a three-phase approach structuring human-LLM collaborative coding through Instruction, Prompt Strategy, and Reliability Check. We evaluated the framework using 72 middle-school mathematics teacher interview transcripts and a human-consensus benchmark. A factorial design compared instructed versus uninstructed prompts, few-shot versus few-shot plus chain-of-thought prompting, and single-pass versus self-consistency coding. Twelve transcripts were used for prompt development and calibration, and 60 for the formal evaluation. In the results, instruction consistently improved agreement. Chain-of-thought further improved agreement under adequate instruction, and self-consistency produced smaller, theme-dependent gains but helped identify unstable decisions for human review. Performance also varied by transcript characteristics. Overall, the framework systematically structures and monitors LLM-assisted qualitative coding while preserving human authority over coding decisions.

Fri 23 OctArtificial Intelligence
The gist
Qualitative coding is a way to organize and interpret complex text data, but doing it with AI has often been unreliable and inconsistent. The authors developed the Smart Coder Framework, which guides humans and AI to work together in three clear steps to make coding more reliable. They tested their approach on real teacher interviews and found that giving clear instructions and thoughtful AI prompting improved agreement with human consensus. This framework helps keep humans in control while using AI to speed up analysis.
Open → 2610.33205v1

Large language models generate coherent survey responses across questions

Simulating Respondents, Not Single Questions: Coherent Survey Generation with Large Language Models

Abstract: Large language models are increasingly used to simulate response distributions in social surveys. Prior work has achieved accurate population-level simulation for individual questions. Real questionnaires, however, ask each respondent a sequence of related questions. A simulated respondent should show coherent preferences across the whole questionnaire, not merely accurate distributions for isolated items. Existing single-item methods cannot accurately reproduce how the same person answers a complete survey. We propose FullRespondent-LLM (FR-LLM), which fine-tunes two specialized LLMs: a marginal model for each item's response distribution and a respondent-level autoregressive model for dependencies across answers. Marginal-Constrained Joint Projection (MCJP) then projects the autoregressive joint distribution onto the set satisfying the item-level marginals learned by the first model. This yields complete questionnaires with realistic cross-item relationships while retaining strong item-level accuracy. On two real-world social survey datasets, FR-LLM more accurately reproduces multi-question response patterns, maintains competitive single-item accuracy, and generalizes better to unseen populations and questions. In a small commercial-survey dataset, we use simulated responses to make pricing and stocking decisions; FR-LLM achieves the highest realized profit.

Mon 28 SeptArtificial Intelligence
The gist
Surveys often ask the same person many related questions, so their answers should make sense together. Existing methods simulate answers for single questions well but fail to keep answers consistent across an entire survey. The authors created a new method that trains two language models: one for each question's answer patterns and another for how answers relate across questions for the same person. Their method better mimics how real people respond to whole surveys and works well even on new populations or questions.
Open → 2609.34828v1

AI moderation matches humans and finds more customer needs

AI-Moderated Interviews for Market Research and Digital Twins Calibration

Abstract: AI-moderated interviews are emerging as a scalable market-research method for generating consumer insights and building consumer "digital twins." Yet it remains unclear whether they match human-moderated interviews or improve on simpler, static data collection methods. In a pre-registered, between-subjects study (N = 317) with three industry partners, we compare AI-moderated (N = 139), human-moderated (N = 24), and static interviews (N = 154). AI moderation matches human moderation in depth, covers more themes, and, holding budget constant, recovers significantly more customer needs than human moderation or static interviews. However, participants sound more emotionally engaged when speaking to a live human. We then create digital twins using interview data and evaluate each twin against the participant's own held-out responses to six real-world marketing stimuli. We find that digital twins created from AI-moderated interviews predict consumer responses better than demographics-only personas. However, the additional richness from AI moderation does not translate into better quantitative predictions compared to static interviews. By analyzing open-ended thoughts generated from humans versus their twins, we find that prediction errors are connected both to differences in (self-reported) thinking styles between twins and humans, and to gaps between training and validation data (i.e., asking questions that are too far out of distribution).

Thu 24 SeptComputers and SocietyArtificial IntelligenceHuman-Computer Interaction
The gist
Finding out what customers want is important but expensive when done by humans. The authors tested AI-led interviews against human and simple surveys with real participants. They found that AI interviews gathered as many insights as human ones and even discovered more customer needs when the budget was the same. However, people felt more emotionally connected talking to humans. The AI data helped make digital models of customers, which predicted responses better than simple demographic profiles but not better than static surveys.
Open → 2609.29143v1