Automatic Model Card Generation Using an LLM
2026-08-25 • Software Engineering
Software EngineeringArtificial Intelligence
AI summaryⓘ
The authors worked on making summaries of machine learning models, called model cards, easier to understand and compare. They created MCTidy, a tool that cleans up and organizes existing model cards into a clear format, and MCGenie, a tool that can write new model cards by looking at model data. Testing showed MCTidy keeps most original information and organizes it well, while MCGenie produces mostly accurate new cards, especially when good supporting info is available. Their work shows that using large language models can help make model documentation simpler and more consistent.
model cardsmachine learning modelslarge language modelsHugging Faceinformation retentionsemantic similarityfactual correctnessmodel documentationnatural language generation
Authors
Tajkia Rahman Toma, Balreet Grewal, Cor-Paul Bezemer
Abstract
Model cards are structured documents that summarize key information about machine learning models to improve transparency, usability, and accountability. However, they often lack a consistent structure, and many models provide no model cards, making comparison and interpretation difficult. This paper presents two contributions. First, we propose MCTidy, an LLM-based approach that reorganizes existing model cards into a standardized template to improve clarity and comparability. Second, we introduce MCGenie, an LLM-based system that generates model cards directly from model repository data. We apply MCTidy to 48 Hugging Face model cards and evaluate information retention, section alignment, hallucination, and stability. Our findings show high information retention with minimal textual loss, accurate section assignment, rare hallucinations primarily in descriptive sections, and strong stability across runs. We assess MCGenie by generating model cards for the same 48 models and assessing semantic similarity, factual correctness, and sensitivity to input resources. The generated model cards achieved high semantic similarity (mean around 0.9); over half were fully correct, and most remaining errors were minor. Generation quality depended strongly on the availability of supporting resources, particularly associated papers. Overall, our findings demonstrate the potential of LLM-based methods to enable scalable, standardized model card documentation.