Transaction prices reflect visual preferences only in limited cases
Do Transaction Prices Reflect Visual Preference? A Diagnostic Study with Multimodal Language Models
Artificial Intelligence
Summary
People often want to know which images or products look nicer based on what people prefer visually. Instead of asking people directly, some think you can guess preferences from the prices items actually sell for online. The authors found that prices don’t always match visual liking because other things affect price, like rarity or demand. They made a new way to test when prices really show visual preferences and how well AI models can figure this out. The study shows prices only sometimes reflect what people visually prefer, so prices aren’t always a reliable guide for this.
What this means in practice
- •For e-commerce data teams: Use diagnostic tests to decide when sales prices can inform visual product ranking models in online stores.
- •For multimodal ai developers: Evaluate and improve AI models that predict human visual preferences using transaction data in limited settings.
Authors
Fengxian Ji, Jingpu Yang, Fan Zhang, Junhong Liang, Qian Jiang, Yuke Li, Jinri Guo, Rongxin Zhu, Zhuohan Xie
Abstract
Existing visual preference evaluation relies on costly human annotations or preference reward models, whereas online marketplace transaction prices are readily available and reflect real purchasing behavior, offering potential low-cost supervision. However, transaction prices conflate visual preferences with non-visual factors such as provenance, demand, and competition, making aggregate accuracy insufficient to distinguish model errors, unreliable price signals, and protocol confounds. To address this identifiability challenge, we introduce MarketImage, a diagnostic framework that identifies when transaction prices reflect visual preferences, where they fail, and whether multimodal language models can recover human-supported visual preference signals. MarketImage comprises three components: (i) MarketImageArt, 266 six-image comparison sets with transaction prices and price-blind human rankings; (ii) MarketImageLens, a tie-preserving protocol that partitions 3,990 within-set image pairs into price-human agreement, conflict, and price-tie regions while separately auditing presentation-order effects; and (iii) MarketValuator, six elicitation methods spanning direct comparison, independent scoring, budget constraints, and multi-agent aggregation to evaluate signal recovery by static visual scorers and three multimodal language models. Experiments show that only 24.2% of image pairs have a non-tied price direction, of which 34.7% agree with the human majority. Within this region, HPSv2 leads at 66.4% Recoverable Accuracy; Qwen's budgeted and aggregated variants trail baselines, while GPT interface gains remain statistically unresolved. Thus, transaction prices encode recoverable visual preferences only under limited conditions and should be treated as uncertain proxies rather than ground truth.