AI system locates and identifies goldsmith marks on silverware

Automated Goldsmith's Mark Retrieval in Silverware

Computer Vision and Pattern RecognitionDigital Libraries

Summary

Identifying goldsmith marks on silverware helps art experts figure out where and when pieces were made, but looking through many examples by hand is slow and hard. The authors created a computer system that finds the marks automatically and uses smart image understanding to recognize them. Their approach works best when the system is taught using a type of learning that doesn’t rely on labels and when it focuses on the mark area. They also share their data and tools to help others use this technology.

What this means in practice

  • For museum digital teams: Automate the detection and matching of goldsmith marks to speed up artifact cataloging and research.
  • For antique appraisal services: Provide faster, AI-assisted verification of silverware authenticity by recognizing maker marks without expert manual comparison.$Commercial implications: Enables launching software that improves the speed and accuracy of evaluating silverware provenance, appealing to appraisal companies.

Authors

Atmik Tiwari, Vincent Christlein, Mark Fichtner, Freya Gohlke, Birgit Schübel, Theresa Witting, Heike Zech, Mathias Zinnen

Abstract

For art historians, goldsmith marks play a critical role in the identification and dating of artifacts. In practice, experts must manually compare a query mark against hundreds of documented examples, a process that is both tedious and highly dependent on specialist knowledge. To address this, we present an AI-assisted retrieval pipeline that combines mark localization with metric-learning fine-tuning across three backbone architectures: an ImageNet-pretrained ResNet-50, a supervised ViT-S/16, and a self-supervised DINOv2 ViT-S/14. We conduct a systematic evaluation of cropping strategies, where we measure the impact of no cropping, manual ground-truth cropping, and learned detection-based cropping, and assess their interaction with each backbone. Our strongest configuration, DINOv2 ViT-S/14 with manual crop and metric-learning fine-tuning, achieves an mAP of 62.63% and a Top-1 accuracy of 73.74%. Our experiments show that self-supervised pretraining and mark localization are the two most impactful factors, with learned cropping recovering the majority of the gain from manual cropping without requiring ground-truth annotations at inference time. To enable reproducibility and adoption in the digital humanities, we release our manually annotated dataset and codebase, and deploy the system via a public web interface.