ChronoQG: Towards a Temporally Expressive and Hop-Bounded Benchmark for Temporal Knowledge Graph Question Generation
Machine Learning
Summary
The authors focus on generating questions from knowledge graphs that include timing information, which previous methods mostly ignored. They created ChronoQG, a new system that builds questions reflecting the timing and order of events in these graphs. ChronoQG produced several datasets with thousands of verified questions involving temporal constraints. Their tests show that current language models have trouble making questions that correctly follow these timing rules, especially when multiple or complex time constraints are involved. This work highlights challenges in generating time-aware questions and provides a new benchmark for future research.
Authors
Xuemeng Liu, Zhengpin Li, Wanpeng Tang, Haotong Xie, Wentao Zhang
Abstract
Knowledge graph question generation (KGQG) aims to generate natural-language questions from structured graph evidence. Existing KGQG benchmarks, however, are mostly built on static knowledge graphs and do not encode the temporal scopes of graph facts. As a result, they cannot evaluate whether generated questions faithfully preserve temporal validity, event ordering, and answer-determining temporal constraints. In this paper, we study temporal knowledge graph question generation (TKGQG), where a generated question must be faithful to both the support subgraph and the temporal constraints required to identify the target answer. We propose ChronoQG, the first temporally expressive and hop-bounded benchmark construction framework for TKGQG. ChronoQG integrates a comprehensive temporal-constraint taxonomy, topology-temporal subgraph sampling, and trace-grounded question generation to construct temporally faithful questions. The framework produces four benchmark datasets from heterogeneous temporal knowledge graphs, totaling 16,011 verified questions. We evaluate representative LLM-based KGQG methods and prompting baselines across diverse TKGQG settings, including temporal-constraint counts, topological templates, and temporal-constraint types. The results show that existing methods struggle to preserve temporal constraints, especially under multi-constraint settings and harder temporal-constraint types. These findings reveal a clear gap between static KGQG and TKGQG, and establish ChronoQG as a challenging testbed for temporally faithful question generation.