Best file formats revealed for finding URLs in research papers
URL Extraction from Scholarly Documents: A Cross-Format Comparative Analysis
Digital Libraries
Summary
Many research papers include web links to extra resources like data or code, but finding these links isn't always easy because papers come in different formats. The authors studied six common formats and combinations to see which worked best for extracting URLs. They created a large dataset to test this and found that the TEXTWAL format was best alone, and combining it with LaTeX gave even better results. Their work shows how important it is to choose the right format when pulling web links from academic documents.
URL extractionscholarly documentsfile formatsLaTeXPDF conversiondata preparationarXivweb linksknowledge graphslink rot analysis
Authors
Rochana R. Obadage, Lamia Salsabil, Sawood Alam, Bipasha Banarjee, William A. Ingram, Edward A. Fox, Jian Wu
Abstract
URLs in scholarly documents link to rich external resources such as datasets, software, publications, and websites. Extracting these URLs is crucial in the data preparation stage of many downstream tasks, such as link rot analysis, web crawling, and building knowledge graphs. However, existing studies often downplay this phase, simply extracting URLs from a single format, usually text directly converted from PDFs. We present a systematic study evaluating URL extraction across six input formats (text with annotation layer, LaTeX, HTML, XML, Markdown, and PNG converted from PDF). To support the evaluation, we compiled a benchmark dataset consisting of 2,338 manually annotated URLs from 200 arXiv papers spanning a wide range of domains over a 33-year period. In addition to evaluating individual file formats, we also compared 63 composite input-format combinations. Our extensive evaluations indicate that TEXTWAL achieves the best performance among single-format inputs, while TEXTWAL+LaTeX achieves the best overall URL extraction performance. The same trend is observed for URLs linking to open-access datasets and software. To further validate these findings, we apply our format-specific URL extraction pipelines to a longitudinal random sample of 364,744 arXiv papers spanning 33 years. We observe a sharp increase in URL density after 2015, along with remarkable differences in URL extraction across file formats over time. Overall, our study highlights the importance of selecting an appropriate format for URL extraction from scholarly documents. The dataset and code are publicly available at: https://github.com/lamps-lab/arxiv-url-bench .