Unified model integrates governance quality provenance and reproducibility in data pipelines
UnespDataLens-RM: A Reference Model for Analytical Data Engineering with Governance, Quality, Provenance, and Reproducibility
Databases
Summary
Data pipelines that prepare information for analysis need to be reliable and well-documented to be trusted. The authors created a general model called UnespDataLens-RM that guides how to build these data processes with important features like governance, quality checks, keeping track of data origins, and making sure the work can be repeated. This model helps organize and improve data workflows from the start. It is designed so others can build real systems or evaluate their pipelines more easily and consistently.
What this means in practice
- •For data engineering teams: Create analytic pipelines with built-in governance and reproducibility features from design, supporting better pipeline management and auditing.
- •For enterprise software developers: Develop software tools that support unified management of data quality, provenance, traceability, and reproducibility within analytic workflows.$Commercial implications: Enables development of governance and auditing modules for enterprise data platforms that comply with industry regulations.
Authors
Ronaldo Celso Messias Correia, Douglas Francisquini Toledo, Camila Tolin Santos da Silva
Abstract
The growing reliance on data in analytical processes and evidence-based decision-making has reinforced the importance of Data Engineering in building pipelines capable of integrating, transforming, validating, and delivering data from heterogeneous sources. However, the reliability of analytical assets depends not only on data processing capabilities but also on mechanisms for governance, quality assurance, provenance, traceability, versioning, and reproducibility throughout their lifecycle. These responsibilities are commonly addressed by different models, frameworks, and operational practices, resulting in methodological fragmentation across the analytical data lifecycle. To address this gap, this article proposes UnespDataLens-RM, a technology-independent reference model that integrates technical-operational processes and cross-cutting capabilities within a unified structure for Analytical Data Engineering. The model aims to support the specification, organization, and evolution of analytical pipelines by incorporating governance, quality, provenance, traceability, and reproducibility from the design stage. Developed following the Design Science Research approach, UnespDataLens-RM comprises eight technical-operational modules, eight cross-cutting modules, complementary dimensions, and a formalized set of artifacts, metrics, and validation criteria. The resulting specification offers a conceptual and methodological framework for future instantiations and empirical evaluations of analytical pipelines designed to be more governable, documented, traceable, auditable, and reproducible.