ModBench: A Pipeline for Building Modelica Benchmark Datasets Mined from Library Repositories
2026-08-17 • Software Engineering
Software Engineering
AI summaryⓘ
The authors created ModBench, a tool that collects and organizes different versions of Modelica models from Git repositories. This helps researchers study how these models change over time and improve tools that work with them. They tested ModBench on the Modelica Standard Library and gathered over 85,000 model snapshots, which are now available for others to use. This work makes it easier to analyze, test, and fix equation-based cyber-physical system models.
ModelicaCyber-physical systemsBenchmark datasetsModel evolutionGit repositoriesSimulation modelsModel snapshotsModelica Standard Library
Authors
Masoud Sadrnezhaad, Martin Sjölund, Adrian Pop, José Antonio Hernández López, Torvald Mårtensson, Dániel Varró
Abstract
Research on equation-based cyber-physical systems modeling languages, such as Modelica, is constrained by the lack of curated benchmark datasets. This limits empirical insight into the evolution and development of models. We address this gap with ModBench, a pipeline that mines Git repositories of Modelica libraries to produce benchmark datasets of model snapshots. The pipeline (1) filters repository commits to retain human-authored, Modelica-relevant revisions; (2) extracts simulation-eligible classes; and (3) builds canonical representations of Modelica classes. For empirical validation, we applied ModBench to the Modelica Standard Library (MSL) and report the resulting dataset, spanning the full commit history (since Modelica language v3), with 85,562 distinct class snapshots, and links enabling traceability to original models and Git metadata. The dataset, its API, and the data generation pipeline are publicly available to support future research on model evolution analysis, compiler testing, and automated model repair or generation.