Fenced Citation-Context Retrieval for Case Law: Temporal Leakage and Degree Control Across Two Jurisdictions
Information RetrievalComputation and Language
Summary
The authors study how legal cases are retrieved based on how future cases cite them, but they point out that using future citations at search time is unrealistic since those citations don't exist yet. They create a method to separate the benefits gained from future citations (which are unavailable at deployment) from those that are legitimately known beforehand. Testing their approach on U.S. and European court datasets, they find that much of the performance gain in naïve methods comes from future information that shouldn’t be counted. Their method performs well without training and matches top systems by only using citations known before the query date.
Authors
Yao Liu, Tien-Ping Tan, Zhilan Liu
Abstract
Incoming citation context-the text with which later cases describe a case when citing it-is a strong signal for legal precedent retrieval, but it is evaluated without a temporal fence: at query time the target precedent has not yet been cited by future cases, so an unfenced method credits itself with evidence unavailable at deployment. Our central contribution is to measure how much: a temporal-admission decomposition that splits the naive relax-the-fence gain into genuine future-citation leakage, legitimate pre-query admission, and an index effect, reporting the phantom fraction-the share that is future evidence. We instantiate this as a controlled deployability audit of incoming-citation retrieval across two jurisdictions, CLERC (US federal, 1.84M documents) and ECtHR-PCR (European Court of Human Rights), adding a zero-training anchor channel that admits only citers dated before the query. On ECtHR-PCR only 14.9% of the naive gain is genuine future evidence; on CLERC the over-credit grows to +4.95 R@1000 as the fence is relaxed. Under the fence, the zero-training channel gains +16.1 R@1000 over BM25 on CLERC, clears a citation-degree control, and reaches a published R@1000 estimate similar to the strongest trained system on ECtHR-PCR (79.56 vs 79.39) at zero training.