Humans show more creative descriptions than AI for hard abstract images
Testing Associative Creativity in Humans and Multimodal LLMs with Difficult-to-Name Visual Stimuli
Computation and LanguageComputer Vision and Pattern Recognition
Summary
The paper looks at how people and advanced AI programs describe strange abstract pictures that are hard to name. It finds that people come up with more varied and original ideas about these images than AI does. When the images get harder to understand, people tend to describe small details instead of whole concepts, while AI struggles more overall. This helps us understand how creativity works in people compared to AI when faced with unusual visuals.
What this means in practice
- •For multimodal ai developers: Improve AI description systems by targeting more human-like creativity metrics in difficult abstract image interpretation.
- •For human-computer interaction designers: Design better tools that account for how humans shift toward low-level visual details when images become complex or noisy.
Authors
Manar Ali, Lisa Gottschalk, Simeon Junker, Lisa Hudalla, Alon Fishman, Joana Cholin, Stefan Kopp, Hendrik Buschmeier, Sina Zarrieß
Abstract
We investigate associative creativity in descriptions of abstract visual stimuli by humans and multimodal large language models (MLLMs). While conceptual association is central to human communication, it remains unclear how humans and models adapt when visual stimuli become increasingly difficult to map onto familiar semantic concepts. We evaluate several state-of-the-art MLLMs and human participants along two dimensions of creativity, originality and effectiveness, using two datasets with controlled manipulations of difficulty. SilCC varies naming difficulty through progressively more complex combinations of abstract shapes, and PixCC varies difficulty through increasing levels of random visual noise. Results show that humans produce more diverse associative descriptions than MLLMs, including greater semantic distance, higher entropy, and more novel expressions relative to reference lexica. As difficulty increases, effectiveness declines in SilCC, and humans increasingly shift from holistic associations to descriptions of low-level visual properties in PixCC.