Papers for

museum digital teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Stable diffusion adapter improves historical art style image generation

The RenAIssance: Caption-Guided Diffusion Models for Historical-Art Visual Media

Abstract: Generative AI can expand cultural collections into prompt-responsive visual media, but historical artworks rarely provide the paired descriptions needed to adapt modern image generators. We present The RenAIssance, a scalable pipeline that turns a collection of historical European artworks into a reusable text-to-image media resource. The pipeline generates scene-level captions with BLIP and uses the resulting approximately 15,000 image--text pairs from DEArt to train a compact LoRA adapter for Stable Diffusion v1.5. The adapter shifts generated images toward the historical-art distribution while retaining the base model's ability to render novel concepts. On historical and contemporary prompts of different lengths, it improves FID to the DEArt reference distribution in every setting with only a modest decrease in CLIP text--image similarity. Qualitative results show that the same adapter can render both period-appropriate subjects and contemporary concepts in a coherent historical visual language. By decoupling a reusable style representation from the base generator, the approach offers a practical building block for controllable historical-art media creation, interactive authoring, and rich-media experiences.

Sat 7 NovComputer Vision and Pattern RecognitionArtificial IntelligenceMachine Learning
The gist
Creating images in the style of historical European art is hard because these artworks usually don’t have descriptions to train AI models. The authors built a system called The RenAIssance that first generates captions for a large set of historical artworks. They then use these captions and images to teach an AI model how to generate new images that look like old art styles while still understanding new ideas. This lets people create art in a historical style on demand, mixing old and new themes naturally.
Open → 2611.10599v1

Museum metadata can confuse AI descriptions more than help

When More Metadata Misleads: Probing Field-Level Grounding in LLM-Generated Museum Descriptions

Abstract: Metadata-conditioned generation is often framed as a data-to-text task in which additional input fields should improve grounding. Museum description generation challenges this assumption because catalogue records are heterogeneous, partially visual, and unevenly documented across geographic regions. We study how metadata composition shapes the generation of artefact descriptions through a two-phase framework. Phase 1 analyses natural metadata variation in a large museum collection, where field count, field type, image availability, and Place co-vary. Phase 2 introduces controlled field probes that hold object identity fixed while varying metadata quantity, field type, and image access. Our results show that metadata quantity alone does not explain generation behaviour. Field type determines whether models ground, omit, or infer information, while images can both support visual grounding and increase unsupported interpretation. More broadly, our findings show that metadata-conditioned data-to-text evaluation should assess field- and claim-level support, rather than relying only on aggregate input coverage or reference similarity.

Thu 22 OctComputation and LanguageArtificial IntelligenceDigital Libraries
The gist
Adding more information fields about museum objects doesn’t always help AI write better descriptions. The authors found that what kind of information is added matters more than how much is included. Sometimes, including images helps AI understand objects better, but it can also lead to making up wrong details. Their work suggests checking each piece of information carefully instead of just looking at overall input or comparing to reference texts.
Open → 2610.31193v1

AI system locates and identifies goldsmith marks on silverware

Automated Goldsmith's Mark Retrieval in Silverware

Abstract: For art historians, goldsmith marks play a critical role in the identification and dating of artifacts. In practice, experts must manually compare a query mark against hundreds of documented examples, a process that is both tedious and highly dependent on specialist knowledge. To address this, we present an AI-assisted retrieval pipeline that combines mark localization with metric-learning fine-tuning across three backbone architectures: an ImageNet-pretrained ResNet-50, a supervised ViT-S/16, and a self-supervised DINOv2 ViT-S/14. We conduct a systematic evaluation of cropping strategies, where we measure the impact of no cropping, manual ground-truth cropping, and learned detection-based cropping, and assess their interaction with each backbone. Our strongest configuration, DINOv2 ViT-S/14 with manual crop and metric-learning fine-tuning, achieves an mAP of 62.63% and a Top-1 accuracy of 73.74%. Our experiments show that self-supervised pretraining and mark localization are the two most impactful factors, with learned cropping recovering the majority of the gain from manual cropping without requiring ground-truth annotations at inference time. To enable reproducibility and adoption in the digital humanities, we release our manually annotated dataset and codebase, and deploy the system via a public web interface.

Thu 17 SeptComputer Vision and Pattern RecognitionDigital Libraries
The gist
Identifying goldsmith marks on silverware helps art experts figure out where and when pieces were made, but looking through many examples by hand is slow and hard. The authors created a computer system that finds the marks automatically and uses smart image understanding to recognize them. Their approach works best when the system is taught using a type of learning that doesn’t rely on labels and when it focuses on the mark area. They also share their data and tools to help others use this technology.
Open → 2609.20509v1

Synthetic aging helps cultural heritage image searches with few old photos

Compensating for Scarce Historical Images in Cross-Domain Cultural Heritage Retrieval Using Synthetic Aging

Abstract: Cultural heritage collections often contain contemporary and historical visual records of the same physical object. Linking these records is difficult because corresponding images may differ in viewpoint, acquisition conditions, color reproduction, framing, resolution, and degradation, while genuine historical images are frequently scarce. This study investigates whether synthetically aged contemporary images can replace or complement missing historical training data in bidirectional instance-level retrieval. Synthetic old-domain images are generated using degradation-oriented transformations. An EfficientNetV2-M model is evaluated on identity-disjoint training, validation, and test sets across three dataset partitions and three training seeds. Mixed real-synthetic training is compared with real-only baselines using proportionally scaled and fixed 300-batch-per-epoch schedules. Complete replacement of genuine historical images reduced bidirectional mean R@1 from 86.56% to 81.27%, showing that synthetic aging does not reproduce the full genuine old-domain variability. Increasing the number of independently generated synthetic variants provided no consistent improvement. Under controlled scarcity, however, synthetic completion improved mean R@1 by 3.69 percentage points at 25% genuine historical coverage and by 2.92 points at 50%, relative to the proportionally scaled real-only baselines. At 75%, the gain decreased to 2.00 points, while performance remained comparable to the complete-real-data reference. Fixed-schedule real-only controls did not reproduce these improvements. The results indicate that genuine and synthetic observations are complementary. Synthetic completion primarily benefits retrieval by extending cross-domain identity coverage rather than by increasing training exposure, with its contribution gradually decreasing as genuine historical coverage increases.

Tue 8 SeptComputer Vision and Pattern Recognition
The gist
It can be hard to match old photos of cultural objects with newer photos because the images look very different. The authors studied whether they could create fake old-looking photos from new ones to help computers learn how to link pictures better. They found that while fake old photos can't fully replace real old ones, they do help when only a small number of real old photos are available. This approach works best when there are some real old photos, improving how well the system finds the same object across time.
Open → 2609.08766v1