Eye-tracking dataset reveals how people look at paintings
AVAtt: A Free-Viewing Eye-Tracking Dataset and Benchmark for Visual Attention on Paintings
Computer Vision and Pattern RecognitionHuman-Computer Interaction
Summary
People look at paintings differently than photos because artists use color, texture, and composition to guide our eyes. The authors created a large dataset that tracks where and when people look at over a thousand paintings from many art styles and centuries. They tested existing models that predict eye attention on these paintings and found these models can improve by considering art-specific details. Their work also shows people look at faces quickly and that art style affects how viewers focus on different parts of a painting.
What this means in practice
- •For museum exhibit designers: Use eye-tracking data from paintings to design displays that guide visitors’ attention more effectively.
- •For digital art app developers: Enhance digital painting tools with models predicting viewer attention to improve user engagement with artworks.$Commercial implications: Enables interactive digital art software that adapts to user gaze patterns for better experience and sales.
Authors
Mohamed Amine Kerkouri, Marouane Tliba, Aladine Chetouani, Alessandro Bruno
Abstract
Saliency and scanpath models are developed almost exclusively on natural photographs, yet paintings are composed by artists to guide the viewer's eye and differ from photographs in texture, colour and abstraction. We introduce \avatt, a free-viewing eye-tracking dataset on \nImages{} paintings spanning sixteen art movements and five centuries, with \nScanpaths{} scanpaths and \nFixations{} fixations from \nParticipants{} participants (\spPerImageMin--\spPerImageMax{} observers per painting). For every painting we release individual scanpaths with fixation timing and eye--screen distance, fixation maps, saliency maps smoothed at one degree of visual angle, temporal saliency videos and rich art metadata. We benchmark classic and deep saliency models against human agreement, revealing substantial headroom for art-aware models, and rank 44 scanpath-similarity measures, including a new language-free family of image-encoder metrics built on ten CNN, ViT and self-supervised encoders. With controls that remove viewing biases, we show that faces are fixated \faceEnrich$\times$ more than expected and within half a second, that local structure draws the eye, that composition rules fitted from image content predict where people look, and that art style shapes how much viewers agree. Dataset, code and a Python package are available at: https://huggingface.co/datasets/Amine-CV/AVAtt.