AI synthetic consumers often fail to predict human visual preferences
Seeing Is Not Perceiving: When Synthetic Consumers Can and Cannot Pretest Visual Marketing
Artificial IntelligenceComputers and Society
Summary
Many marketers use AI models as fake customers to test ads and designs before showing them to real people. The authors found that these AI models do see the images but don’t truly understand the meaning or appeal like humans do. In tests of six marketing visuals, the AI only matched human reactions a few times and sometimes gave opposite results. Adding extra hints helps somewhat, but AI still misses much of the variety in how real people respond. This suggests that marketers should not rely only on AI and still need human feedback for visual marketing.
What this means in practice
- •For marketing teams: Know when to trust AI simulations for early visual design tests versus when human panels remain essential.
- •For advertising agencies: Use AI insights cautiously to complement but not replace human consumer feedback on logos and packaging.
Authors
Yi-Lin Tsai, Yung-Hsiu, Lai
Abstract
Marketers now deploy generative AI agents as synthetic consumers to pretest visual assets such as logos, packaging, and advertising at a fraction of human-panel cost. However, this procedure assumes that a model seeing a visual cue can also perceive its consumer meaning, which is largely untested. We stress-test the assumption using six canonical visual marketing experiments, varying the two levers managers control: model generation (GPT-4o-mini vs. GPT-5.4-mini) and input format (plain text vs. JSON). Every resulting configuration passed the manipulation checks; however, none of the configurations reproduced more than two of the six human effects, and the remainder were nonsignificant. The one exception was a significant reversal of the human pattern. Providing conceptual or empirical evidence through in-context learning steers average responses toward the human effect. Yet steering has a limit: even when it succeeds, a configuration reproduces less than half of the natural spread of human responses and so understates consumer heterogeneity. We integrate these results into an AI governance protocol (Calibrate, Intervene, Deploy) that delineates when synthetic consumers can responsibly screen creatives and when human panels remain necessary.