Stable diffusion adapter improves historical art style image generation

The RenAIssance: Caption-Guided Diffusion Models for Historical-Art Visual Media

Computer Vision and Pattern RecognitionArtificial IntelligenceMachine LearningMultimedia

Summary

Creating images in the style of historical European art is hard because these artworks usually don’t have descriptions to train AI models. The authors built a system called The RenAIssance that first generates captions for a large set of historical artworks. They then use these captions and images to teach an AI model how to generate new images that look like old art styles while still understanding new ideas. This lets people create art in a historical style on demand, mixing old and new themes naturally.

What this means in practice

  • For digital artists: Create images that blend historical art styles with modern concepts using a specialized adapter trained on European artworks.
  • For museum digital teams: Generate period-appropriate visual media that enriches cultural collections and public exhibits by applying the trained style adapter.

Authors

Zhanyu Tuo

Abstract

Generative AI can expand cultural collections into prompt-responsive visual media, but historical artworks rarely provide the paired descriptions needed to adapt modern image generators. We present The RenAIssance, a scalable pipeline that turns a collection of historical European artworks into a reusable text-to-image media resource. The pipeline generates scene-level captions with BLIP and uses the resulting approximately 15,000 image--text pairs from DEArt to train a compact LoRA adapter for Stable Diffusion v1.5. The adapter shifts generated images toward the historical-art distribution while retaining the base model's ability to render novel concepts. On historical and contemporary prompts of different lengths, it improves FID to the DEArt reference distribution in every setting with only a modest decrease in CLIP text--image similarity. Qualitative results show that the same adapter can render both period-appropriate subjects and contemporary concepts in a coherent historical visual language. By decoupling a reusable style representation from the base generator, the approach offers a practical building block for controllable historical-art media creation, interactive authoring, and rich-media experiences.