Do Generative Models Keep Time? A Time-Aware Evaluation of Synthetic Sequential Tabular Data
Machine Learning
Summary
The authors explain that synthetic data used for sharing private sequential information can have hidden problems like incorrect timestamps or unrealistic entity behavior. Traditional evaluation methods miss these issues because they treat data records independently without considering time. To fix this, the authors propose a new way to check synthetic data that looks at how time works in the data and how entities change over time. They tested their method on various models and datasets and found it gives very different results from conventional checks, showing it's important to evaluate data with time in mind.
Authors
Kiwan Kwon, Kangmin Kim, Hojin Lee, Yeseong Jung, Hyeongwoo Kong, Vamsi K. Potluru, Saerom Park, Yongjae Lee
Abstract
Synthetic sequential tabular data are increasingly used for privacy-preserving data sharing, yet a generator can reproduce every marginal and every foreign-key relationship while emitting timestamps that run backwards or repeat, and while sending entities along paths that no real entity followed. Conventional tabular evaluation, which pools records into static distributions, is blind to such failures. We present a taxonomy-guided evaluation protocol for temporal fidelity, in which the applicable measurements are determined by the data rather than fixed in advance. Each dataset is first characterized along four properties: how time is represented, whether observations are regularly sampled, whether trajectories are mutually dependent, and how the schema links entities to their histories. These properties determine which evaluation dimensions are meaningful. The protocol then measures timestamp validity, cross-sectional structure at aligned time points, within-entity dynamics, and time-varying relational structure, and recasts utility and privacy evaluation over trajectories rather than isolated rows. We apply the protocol to eight generative models across thirteen datasets spanning six domains. Rankings under conventional evaluation disagree substantially with those obtained under temporal evaluation, and the resulting failures are architecture-coherent rather than random. Temporal fidelity must therefore be measured on the time axis itself, rather than inferred from pooled record distributions.