Indian legal case retrieval improves with rhetorical role focus
A Study of Prior Case Retrieval Using Lexical, Semantic, and Rhetorical Role Information in Indian Legal Documents
Information Retrieval
Summary
Finding relevant past legal cases in Indian judgments is hard because simply matching words or laws isn’t enough to find truly useful cases. The authors show that looking at different parts of legal documents, like facts and arguments, works better than just reading the whole text or relying on laws mentioned. They also found that some advanced AI ranking methods don’t perform well on official tests. They propose a new four-step method that significantly improves identifying relevant prior cases.
What this means in practice
- •For legal technology developers: Build better search tools to find relevant prior cases in Indian legal documents by focusing on document sections like facts and arguments instead of full text.
- •For information retrieval engineers: Improve ranking algorithms for legal document retrieval by using query splitting and validation fusion techniques shown effective in this Indian legal context.
Authors
Sayed Ayaan Ahmed Sha, Sangeetha Sivanesan, Anand Kumar Madasamy, Navya Binu, Aniket Mani, Rishu Kumar
Abstract
For retrieving prior cases in Indian legal judgments, the problem involves distinguishing relevant legal facts from mere lexical similarities because a prior case that shares a statute with the query judgment is not necessarily relevant. In this paper, we describe an empirical evaluation of three consecutive designs of retrieval systems for the IL PCR(Indian Legal Prior Case Retrieval) task. We demonstrate that the combination of the rhetorical roles (Fact, Ratio Of The Decision, Precedent, Argument, Statute) is better than either using individual roles or performing full text retrieval, that statute similarity is not discriminative, and that a legal entailment reranker with training data produced by an LLM is much worse in terms of official evaluation than its internal validation score. Two rankers, with excellent internal MRR up to 0.97, performed poorly in terms of official evaluation (MRR as low as 0.14). Thus, we propose a new design with query disjoint splitting and frozen validation fusion. The result is a four stage pipeline with Micro F1 0.2549, MRR 0.6081, and nDCG@10 0.4187.