DelistBench: Evaluating Search-Enabled LLMs for Auditable Corporate-Event Database Completion
2026-08-24 • Computation and Language
Computation and Language
AI summaryⓘ
The authors present a way for financial institutions to independently check if important corporate events like delistings are missing or wrong in vendor databases. They introduce a task called Search-to-Record, where language models use public web data to recreate event records, and provide DelistBench, a benchmark with 1,200 delisting records to test this. Their tests show that giving models web access greatly improves accuracy, and even cheaper systems can get close to the best results. The authors suggest using risk-based checks to focus human review on the most uncertain cases, helping improve data quality efficiently.
corporate eventslarge language modelsdatabase assurancedelisting announcementsSearch-to-RecordDelistBenchweb retrievalrisk-based triageevent record accuracyfinancial data validation
Authors
Xuan Yao, Li Shuping, Dai Yang, Zhou Yi, Ke-Wei Huang
Abstract
Financial institutions need an independent way to detect missing, stale, and misclassified corporate-event records in vendor databases. We introduce Search-to-Record, a database-assurance task in which search-enabled large language models reconstruct institution-defined event records from public sources for a known security universe and historical cutoff, and DelistBench, a 1,200-record benchmark for security-level delisting announcements. We evaluate five models in paired closed-book and web-enabled conditions. Web access raises announcement-date accuracy within seven days by 34.0 to 48.0 percentage points and event-status accuracy by approximately 2.8 to 21.7 points; the best system achieves 81.5% overall joint accuracy within seven days. Economy web systems achieve 75.9-78.3% overall joint accuracy within seven days at 4.5-6.6% of the API cost of the most expensive web system. Risk-based triage identifies low-error subsets, although the highest-coverage operating point still sends 27.3% of the balanced test set to review. The evaluation identifies web retrieval as the main source of timing gains and shows that low-cost systems can approach the best system's accuracy. Together, Search-to-Record, DelistBench, and the evaluation provide concrete deployment guidance: calibrate triage to local event prevalence and market mix, preserve positive-event recall, and route positive and ambiguous cases to targeted review.