LexKairos: Benchmarking Legal Temporal Capabilities in LLMs
2026-08-10 • Computation and Language
Computation and Language
AI summaryⓘ
The authors created LexKairos, a test set to see how well large language models (LLMs) understand time-related rules in Chinese law. LexKairos includes tasks about knowing legal time rules, understanding timelines in court cases, and reasoning about how laws and cases relate in time. They tested eight different LLMs and found that while Gemini-3-Flash did best, even it struggled with precise and complex time-related legal questions. This shows that current LLMs still have trouble with legal time knowledge and reasoning. The authors have shared their data and code publicly.
large language modelslegal AItemporal reasoningChinese lawstatutory knowledgecase modelingChain-of-Thoughttime-sensitivejudicial casesbenchmark
Authors
Chenyang Li, Zejia Feng, Yuqin Huang, Yuxiao Ye, Huiyuan Xie
Abstract
Large language models (LLMs) have demonstrated strong performance across a wide range of legal tasks. In legal practice, time is a critical concept that governs the validity of statutes, the progression of legal cases, and the enforcement of procedural deadlines. However, legal temporal capabilities remain underexplored in existing legal AI benchmarks. To address this gap, we propose LexKairos, a comprehensive benchmark for evaluating the temporal capabilities of LLMs in the Chinese legal context across three dimensions: statutory temporal knowledge, case temporal modeling, and statute-case temporal reasoning. LexKairos comprises nine sub-tasks drawn from real-world Chinese judicial cases and statutes. We conduct systematic evaluations of eight LLMs under multiple inference settings, including vanilla, Chain-of-Thought (CoT), and thinking modes. Our results show that Gemini-3-Flash achieves the strongest overall performance, yet even the best-performing model exhibits notable limitations on tasks demanding precise time-sensitive statutory metadata recall or complex reasoning in time limits, indicating that legal temporal knowledge and reasoning remain open challenges for current LLMs. Data and code are available at https://github.com/thunlp/LexKairos.