Improving Information Extraction with Learned Queries
2026-08-31 • Computation and Language
Computation and Language
AI summaryⓘ
The authors found that when computers try to pull out information, the way questions are asked is just as important as how big or smart the model is. They showed that better questions alone improved results more than just using bigger models. To make better questions, they created methods called List of Questions (LoQ) and FeedQ that keep improving questions based on how well the computer extracts info. These improved questions help smaller models do as well as or better than much larger ones. They also shared a big set of these optimized questions to help others work on this idea.
information extractionquestion designlarge language models (LLMs)F1-scoreList of Questions (LoQ)FeedQfine-tuningclinical benchmarksmodel optimizationdataset release
Authors
Omar Sharif, Soroush Vosoughi, Nikhil Singh
Abstract
When information extraction fails, a natural instinct is to improve the model doing it: for example, by scaling it up or refining its reasoning. In this paper, we show that another part of the pipeline matters at least as much: the queries used to elicit this information. Across four clinical benchmarks and five LLMs, improving the question design alone raises performance by 18.6 F1-score points, i.e. more than using larger extraction models. To make such question design learnable, we introduce List of Questions (LoQ), which generates document-specific question sets, and FeedQ, a feedback-driven optimization method that iteratively refines questions against extraction outcomes. The resulting optimized questions can be used to train lightweight generators: with fine-tuning, 4B-parameter models match or outperform expert-derived baselines and substantially exceed the performance of much larger untuned models. We release a dataset of 12,820 optimized questions to support a broader shift in information extraction research toward treating question design as a first-class problem.