Retrieval has limited effect on patent claim amendment by AI models
Grounded Revision vs. Prior Injection: Probing Retrieval-Augmented Patent Claim Amendment
Computation and Language
Summary
When AI models help rewrite patent claims, it’s unclear if they truly use relevant past examples or just copy templates. The authors studied thousands of patent cases where the 'correct' changes are clearly defined by examiners. They tested multiple large language models using different ways to retrieve related information. Their results show that these models do not consistently use retrieved data as true grounding; instead, retrieval had small and inconsistent effects on the claim revisions.
What this means in practice
- •For patent drafting teams: Improve AI-assisted tools by understanding how retrieval affects claim amendment to better support patent rewriting workflows.
- •For legal tech developers: Develop retrieval strategies in legal document automation acknowledging the limited grounding effect found in current LLMs for patent claim edits.
Authors
Josepha Michiko Leo, Hyun-seok Min, Yehoon Jang, Irvan Zidny, Jin-Woo Chung, Sungchul Choi
Abstract
Retrieval-augmented generation is widely used in professional writing, yet whether retrieval grounds revision or merely injects templates is rarely tested where "correct" has a definable meaning. Patent claim amendment supplies that signal: the examiner names the attacked limitation and cites prior art, providing per-case ground truth. We release three artifacts: (i) a corpus of 7,385 USPTO prosecution cases with XML-aligned pre/post claims, rejection, and cited prior art; (ii) a seven-probe battery comparing random and structural-match retrieval as two policies under a fixed prompt scaffold; (iii) a deterministic five-channel metric (C1-C3 and C5 in main, C4 supplementary) requiring no LLM evaluation. Across 9,600 pre-registered calls on four frontier LLMs (Claude Sonnet 4, Claude Haiku 4.5, GPT-5.4, GPT-4o-mini), no tested model exhibits detectable classical prior-injection behavior; retrieval effects are small and direction-inconsistent between random and structural retrieval, and the null is unchanged under a dense (semantic) retriever, across retrieval depths k in {1,3,5,10}, and under a paraphrase-sensitive grounding metric. Revision locality reveals a model-specific difference that the template channel misses. The four-cell taxonomy, which we treat as exploratory, leaves the prior-injector cell unoccupied.