The GELATO Dataset for Legislative NER
Computation and Language
Summary
The authors created GELATO, a special dataset of U.S. legislative bills with detailed labels to identify important parts in the text. They trained different transformer models like BERT and RoBERTa to recognize these first-level labels. Then, they used large language models (LLMs) with smart instructions to predict the second-level details. Their results showed RoBERTa worked better than BERT, and combining transformers with LLMs could help with analyzing legislative documents in the future.
Authors
Matthew Flynn, Timothy Obiso, Sam Newman
Abstract
This paper introduces GELATO (Government, Executive, Legislative, and Treaty Ontology), a dataset of U.S. House and Senate bills from the 118th Congress annotated using a novel two-level named entity recognition ontology designed for U.S. legislative texts. We fine-tune transformer-based models (BERT, RoBERTa) of different architectures and sizes on this dataset for first-level prediction. We then use LLMs with optimized prompts to complete the second level prediction. The strong performance of RoBERTa and relatively weak performance of BERT models, as well as the application of LLMs as second-level predictors, support future research in legislative NER or downstream tasks using these model combinations as extraction tools.