Museum metadata can confuse AI descriptions more than help

When More Metadata Misleads: Probing Field-Level Grounding in LLM-Generated Museum Descriptions

Computation and LanguageArtificial IntelligenceDigital LibrariesMachine Learning

Summary

Adding more information fields about museum objects doesn’t always help AI write better descriptions. The authors found that what kind of information is added matters more than how much is included. Sometimes, including images helps AI understand objects better, but it can also lead to making up wrong details. Their work suggests checking each piece of information carefully instead of just looking at overall input or comparing to reference texts.

What this means in practice

  • For museum digital teams: Improve quality control for AI-generated artefact descriptions by verifying which metadata fields reliably produce accurate content.
  • For ai content developers: Design metadata inputs for large language models to better align generated text with visual and catalogue data, reducing misleading outputs.

Authors

Clemencia Siro, Laura Hollink

Abstract

Metadata-conditioned generation is often framed as a data-to-text task in which additional input fields should improve grounding. Museum description generation challenges this assumption because catalogue records are heterogeneous, partially visual, and unevenly documented across geographic regions. We study how metadata composition shapes the generation of artefact descriptions through a two-phase framework. Phase 1 analyses natural metadata variation in a large museum collection, where field count, field type, image availability, and Place co-vary. Phase 2 introduces controlled field probes that hold object identity fixed while varying metadata quantity, field type, and image access. Our results show that metadata quantity alone does not explain generation behaviour. Field type determines whether models ground, omit, or infer information, while images can both support visual grounding and increase unsupported interpretation. More broadly, our findings show that metadata-conditioned data-to-text evaluation should assess field- and claim-level support, rather than relying only on aggregate input coverage or reference similarity.