AI summaryⓘ
The authors address a problem in generating patent claims, where the structure requires each claim to depend correctly on a parent claim, forming a hierarchical tree. Traditional methods can't handle this well because they generate claims one token at a time without enforcing these hierarchical dependencies. The authors propose a method called SPG that predicts relationships between claims (which claim depends on which) during generation and uses special training techniques to improve the model's understanding of this structure. Their method improves the accuracy of predicting parent links and the consistency of claim dependencies compared to previous models. Expert evaluation confirms these improvements.
autoregressive decoderpatent claim generationhierarchical constraintsdependency forestpointer networkscope regularizerself-generated candidatesantecedent consistencyLlama-3-8B-Instructtopology prediction
Abstract
Autoregressive decoders emit flat token sequences and cannot enforce hierarchical constraints across output segments, a limitation that becomes acute in patent claim generation, where a claim set forms a dependency forest whose scope must narrow monotonically with depth. Topology and content are mutually dependent: a dependent claim's wording must reflect its parent's scope, yet the parent must be chosen before that wording exists, so neither post-hoc parsing nor grammar-constrained decoding suffices. We propose SPG (Structure-aware Patent Generation), which predicts topology inside the autoregressive pass. A pointer head selects each dependent claim's parent, and its gradients, together with a depth-adaptive scope regularizer, reshape the shared decoder's representations during training. A second stage then applies a violation-weighted preference objective over self-generated deficient candidates, supplying the negative signal that granted-patent corpora lack. On HUPD-DCG, SPG on Llama-3-8B-Instruct recovers 79.0\% of gold parent links, a quantity its training reward never supervises, and raises antecedent consistency from 0.292 to 0.478 over a supervised baseline of equal scale, with expert evaluation corroborating these gains.