Can MLLMs Critique Like Humans? Evaluating Open-Ended Aesthetic Reasoning in Multimodal Large Language Models
2026-06-29 • Computation and Language
Computation and Language
AI summaryⓘ
The authors studied how well multimodal large language models (MLLMs) can write aesthetic critiques of photos, comparing them to human critiques from Reddit. They found that common methods measuring similarity to human critiques can be misleading, as models tend to write longer, more detailed, and more repetitive critiques than humans. The models often produce a general style rather than image-specific insights. The authors suggest that current evaluation methods reward fluency and coverage rather than the focused and selective nature of human critique, highlighting challenges in assessing open-ended AI-generated aesthetic reviews.
Multimodal large language modelsAesthetic critiqueReference-based similarityReddit Photo Critique DatasetLexical metricsEmbedding cosine similarityOpen-ended generationImage groundingEvaluation metricsPrompt conditioning
Authors
Sajjad Ghiasvand, Maryam Amirizaniani, Haniyeh Ehsani Oskouie, Mahnoosh Alizadeh, Ramtin Pedarsani
Abstract
Open-ended aesthetic critique is a challenge for multimodal large language models (MLLMs): unlike multiple-choice aesthetic benchmarks, it has no single correct answer, and most aesthetic evaluation has measured models against numeric scores rather than the written critiques people actually give. We evaluate MLLM critiques against ranked human references and ask whether they are close to human ones. Using the Reddit Photo Critique Dataset, we score five open-weight MLLMs against multiple ranked human critiques per photo with reference-based similarity metrics, under six prompt conditions that disentangle persona framing, aspect hinting, length control, and single- versus multi-pass generation, and add an image-grounding control that feeds each model the wrong photograph. We find that reference-based similarity gives a misleading picture. Stricter lexical and learned metrics show only weak alignment with human critiques, while a coarse embedding cosine reports broad topical overlap that the grounding control traces to a stable house style rather than image-specific observation. Behaviorally, the models diverge from humans in consistent ways the scores do not surface: even under a length cap they write two to three times as much, cover nearly every aesthetic aspect where humans are selective, engage each aspect more uniformly and at greater depth, and repeat themselves across critiques of the same photo where humans vary. We argue that reference-based similarity rewards a fluent, comprehensive critique style rather than the selectivity and specificity of human critique, and discuss implications for evaluating and training open-ended multimodal generation.