Neural graphs improve patent search on very long documents
From Rules to Neural Graphs: Scalable Structured Prediction for Patent Prior Art Search
Information RetrievalComputation and Language
Summary
Patent documents can be very long, making it hard for computers to search them effectively because most methods only look at part of the document. The authors created a new neural parser that builds detailed graphs representing inventions directly from the full patent text. This method is faster and more accurate than older rule-based parsers, handling long documents without needing retraining. Using these neural graphs helps find relevant cited patents more reliably in large patent searches.
What this means in practice
- •For patent office teams: Use neural graphs to improve accuracy and speed when retrieving prior art from extremely long patent documents.
- •For legal tech developers: Build patent search tools that handle full-length patent texts efficiently, enhancing recall in citation and prior art retrieval tasks.$Commercial implications: Enables creation of advanced patent search products that better identify relevant citations, attracting IP law firms and patent services.
Authors
Nikolai Zenovkin, Sebastian Björkqvist
Abstract
Patent search requires processing documents routinely exceeding tens of thousands of tokens. Most neural retrieval approaches operate on truncated inputs, limiting their effectiveness. Graph-based retrieval addresses this by representing each patent as a structured invention graph, but constructing these graphs relies on brittle rule-based parsers. We present the neural parser, which adapts biaffine attention from dependency parsing to predict invention graphs directly from patent text. Our local biaffine attention restricts pairwise scoring to a sliding window, reducing complexity from $O(n^2)$ to $O(n \cdot w)$. Since local and global scoring share the same weights, the model trains on short sequences and deploys on documents exceeding 40,000 tokens without retraining. Distilled from 1 million rule-parsed documents, it surpasses its teacher at 3$\times$ lower inference cost: neural graphs improve citation recall by 0.5% on short queries and 1.1% on full documents in a downstream Graph Transformer retrieval system.