Paragraph level precedent retrieval improves UK legal research
UK-PRBENCH: A Paragraph-Level Precedent Retrieval Benchmark for United Kingdom Case Law
Information RetrievalComputation and Language
Summary
Legal cases often contain many paragraphs, but only some are relevant when searching for similar cases. Most current systems look at whole documents, which can be inefficient and imprecise. The authors created UK-PRBench, a new test set for retrieving relevant paragraphs within UK case law. They tested current best methods and found that this detailed task is still difficult, showing there is room to build better tools for lawyers.
What this means in practice
- •For legal technology developers: Develop and evaluate finer-grained tools that find relevant paragraphs in UK legal cases to assist lawyers with precise precedent identification.
- •For legal document analysts: Improve automated analysis by focusing on important paragraphs rather than whole judgments, increasing efficiency in legal information management.
Authors
Damith Premasiri, Tharindu Ranasinghe
Abstract
Prior case retrieval (PCR) aims to identify precedent cases relevant to a given query case. Existing PCR benchmarks and methods predominantly operate at the document level, treating entire judgments as the unit of relevance. This formulation is suboptimal for legal practitioners, as judgments address multiple legal issues and only a small subset of paragraphs is relevant to a particular query. Addressing this gap, we introduce UK-PRBench, a benchmark for paragraph-level precedent retrieval in UK case law, constructed from judgments obtained from the UK National Archives and covering a broad range of UK courts and tribunals. Furthermore, we evaluate state-of-the-art retrieval models and establish baseline results. Our experiments show that paragraph-level precedent retrieval remains challenging for current retrieval approaches, highlighting substantial room for improvement. UK-PRBench provides a standardised benchmark for evaluating fine-grained precedent retrieval and advancing retrieval systems for the UK legal domain.