Papers for

digital library developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Multimodal memory cues aid finding scientific papers without exact details

SciToT: Multimodal Tip-of-the-Tongue Retrieval for Scientific Papers

Abstract: Researchers frequently encounter difficulties re-finding scientific papers when they remember aspects of the content but cannot recall reliable identifiers, such as titles or author names. This type of re-finding task, known as tip-of-the-tongue (ToT) retrieval, is particularly challenging for scientific documents, where recollections may involve textual information and visual elements, including figures, diagrams, and plots. Existing ToT retrieval benchmarks do not address scientific search. We introduce a multimodal benchmark for scientific ToT retrieval, comprising 108,512 open-access scientific documents across five domains and 11,499 synthetically generated queries. To construct the benchmark, we propose a scalable query elicitation pipeline that independently extracts textual and visual memory cues from source documents and uses large language models to generate queries resembling incomplete human recollections. We characterize scientific ToT information needs through an empirical analysis of publicly available re-finding requests and use the outcomes to assess the generated queries using human annotations and automated quality control. The benchmark includes lexical, dense, late-interaction, and visually grounded retrieval baselines. We release the dataset, extracted document representations, generated memory cues, evaluation splits, and accompanying generation and retrieval pipelines under an MIT license to facilitate research on multimodal scientific paper re-finding.

Wed 4 NovInformation Retrieval
The gist
People often struggle to find scientific papers when they remember parts of the content but not titles or author names. This problem is harder with scientific papers because memories can involve text and images like figures or charts. The authors created a big test set of scientific papers and realistic search queries that mimic how people remember. They also made computer programs to help find papers using both the text and images that people recall. Everything is shared openly to help others build better tools to rediscover scientific articles.
Open → 2611.05829v1

Dhārā bench measures search for questions on indian knowledge texts

Dhārā Bench: An Evidence-Oriented Retrieval Benchmark for Indian Knowledge Systems

Abstract: Dhārā Bench is a retrieval benchmark for Indian Knowledge Systems (IKS), with 325 questions in 12 categories and 60,121 text chunks from 15 source groups. The categories were developed based on 4,500 questions submitted to the Dhārā application; the evaluation set combines corpus-aligned application queries with expert-contributed questions. The annotations comprise 7,578 question-chunk relevance links involving 6,847 distinct corpus chunks. For 34 composite and multi-hop questions, 401 conjugate groups specify alternative sets of passages that jointly support each question. Chunk identifiers and hierarchical references connect the judgments to their source texts. The paper describes corpus preparation, annotation, and validation, and reports a fixed BM25 baseline implemented with Pyserini with $k_1=0.9$ and $b=0.4$. Across all questions, BM25 achieves MRR@20 of 0.315, Recall@10 of 0.182, and nDCG@10 of 0.205. On the group-annotated subset, complete-evidence scores are CE_Recall@20 = 0.294 and CE_MRR@20 = 0.141. The category results expose particularly weak verse retrieval, while group scores reveal gaps between finding individual passages and completing their support. The corpus and evaluation code are available at https://github.com/Avinash2236/Dhara-Bench.

Mon 2 NovInformation RetrievalComputation and Language
The gist
Finding reliable information in ancient Indian texts can be tough. The authors created Dhārā Bench, a test with 325 questions and thousands of text pieces from Indian Knowledge Systems. They measured how well a standard search method, BM25, could find relevant passages to answer these questions. The results reveal strengths and weaknesses in retrieving different types of text, especially verses, and show gaps between identifying needed documents and fully supporting answers. They made the data and tools publicly available to help improve search in these texts.
Open → 2611.03196v1

Crosschapterqa dataset helps ai answer book questions spanning chapters

Reading Between the Chapters: A Graph-Free RAG Rebuilt With AI and Human Feedback

Abstract: Many questions about a book can only be answered by connecting passages chapters apart, such as a cough heard three times across a horse's life. We introduce CrossChapterQA, 1,853 such questions over 14 public-domain books. Each question lists the two to six chapters it needs (spanning half the book at the median) and the relation linking them, cites evidence from each, and has two independent reference answers graded blind. We use it to build LadderRAG, which indexes a book as passages, summaries and book-level cards for about $0.41 and answers a question for under a cent. Its first version failed mostly at retrieval. We redesigned it four times, steered by an LLM judge's verdicts and our analyses of failed answers: a design-level counterpart of RLAIF and RLHF in which feedback changes the architecture, not model weights. We also report the two redesigns that made answers worse. On the 191 questions its first version got wrong, the final version retrieves over twice as many required chapters on held-out books (recall 0.36 to 0.79), and ten annotators rate its answers 6.5/10 on average (764 ratings).

Sat 31 OctArtificial Intelligence
The gist
Answering questions about a book often requires linking information from chapters far apart, like noticing a sound happening multiple times in a character's life. The authors created CrossChapterQA, a set of questions that need this kind of cross-chapter reasoning across 14 books. They built LadderRAG, a tool that uses summaries and passages instead of graphs to find the right chapters and answer questions. After several redesigns guided by AI and human feedback, the final version retrieves more relevant chapters and produces better answers.
Open → 2611.00752v1

How bert cross-encoders understand relevance between queries and documents

How BERT-based Cross-Encoders Model Relevance in Information Retrieval?

Abstract: Cross-encoders are the most effective but also the least efficient models in Information Retrieval. Despite their success, the mechanisms underlying their relevance predictions remain poorly understood, particularly how they process to match queries and documents. This lack of interpretability limits their optimization, can lead to suboptimal retrieval, especially out-of-domain, and raises efficiency concerns for over-parameterized models. To address these challenges, this paper investigates two under-explored aspects of cross-encoder behavior. First, to understand the relevance prediction process, we trace the flow of information at the different stages of the process and between different input segments. We study the nature of the signals that compose this flow and that allow cross-encoders to predict the relevance of a passage to a query. Our work particularly focuses on the matching performed by the cross-encoders between queries and documents. Second, we explore how cross-encoders behave with out-of-domain documents and/or queries. Through a series of interpretability experiments, we evaluate three BERT-based cross-encoder models, varying in size and pre-training strategies, across both in-domain and out-of-domain datasets. Overall, this study both consolidates what is known about how cross-encoders model relevance and extends it by exploring the role of matching signals.

Fri 30 OctInformation Retrieval
The gist
Matching queries with relevant documents is important but sometimes hard to understand for advanced AI models like BERT-based cross-encoders. This paper looks inside these models to see how they figure out if a document answers a question, including how they handle different types of queries and documents they weren’t trained on. The authors analyze multiple versions of these models to identify the key signals they use for matching. Their results help explain why these models perform well and how they might be improved or better applied in practice.
Open → 2610.41425v1

Language model system improves document search without keyword matching

Retriever-Free Retrieval: Navigating Documents with Decision Models

Abstract: While language models have become dramatically more capable, retrieval still largely relies on lexical matching or similarity in fixed embedding spaces. This creates a capability asymmetry: what the retriever fails to find, even the strongest generator cannot recover. We propose DecisionRAG, which brings the capabilities of pretrained language models to retrieval using decision models that jointly evaluate candidate branches in a single forward pass, without generating any tokens, and show that judging candidates in a shared context makes their relevance estimates comparable. Like a person browsing a library, DecisionRAG navigates the corpus's inherent structure from broad topics down to individual passages, deciding at each level which branches are worth exploring. Without task-specific training or a first-stage retriever, DecisionRAG outperforms dense retrieval with an 8B reranker by 13.8 R@1 points on MultiDoc2Dial and 17.4 Hit@1 points on Natural Questions. Across four benchmarks, including a corpus of 2.68 million Wikipedia paragraphs, retrieval takes just 0.7-1.8 seconds per query.

Mon 26 OctArtificial Intelligence
The gist
Finding relevant information in large collections of documents usually depends on matching words or fixed patterns. This paper introduces DecisionRAG, a method that uses language models more like a person browsing a library, moving from broad topics to specific passages and deciding what to explore next without needing special training or initial keyword searches. The authors found that DecisionRAG can find better information faster than existing methods on several test sets. This approach helps language models better navigate and retrieve useful content from huge document collections.
Open → 2610.35868v1

RefinedGAW dataset boosts coverage of German academic web texts

RefinedGAW: A Refined German Academic Web Dataset

Abstract: The largest open German text datasets are built from broad web crawls, which provide scale but limited coverage of specific domains. We introduce RefinedGAW, a processed and annotated resource derived from 26 semi-annual crawls of the German Academic Web, totaling 122 million documents and 314 billion tokens, with full crawl and capture history of every text. Documents labeled as academic research account for 84% of German RefinedGAW tokens across HTML and PDF. At 87.6 billion tokens, this exceeds the 64.9 billion tokens in the German subsets of FineWeb2 and FinePDFs, despite their almost five-times-larger combined size. RefinedGAW also contains substantial amounts of educational, math-heavy, and expert-oriented content with minimal commercial bias. Across HTML and PDF, 88% of German RefinedGAW documents have no near-duplicate candidate match in the three evaluated reference datasets, supporting complementary coverage. We release a queryable metadata index for fine-grained data selection and a rehydration tool for retrieving and processing source documents.

Sun 25 OctComputation and LanguageInformation Retrieval
The gist
Large collections of German text from the internet often miss specific academic and educational content. The authors created RefinedGAW, a huge dataset focused mainly on German academic texts gathered from many web crawls over time. This dataset has more academic content than similar collections and includes a detailed history and metadata for fine searching. It also has very few repeated texts compared to other datasets, offering fresh and diverse materials. The authors provide tools to access and use the documents effectively.
Open → 2610.34572v1

Directional feedback improves search queries for better retrieval results

Directional Posterior Feedback Fusion for Query Reformulation in Dense Retrieval

Abstract: Dense retrieval has become a fundamental paradigm for modern information retrieval, supporting efficient semantic search across a wide range of applications. Its effectiveness, however, relies critically on query embeddings that adequately capture users' information needs, which can be difficult for short, ambiguous, or underspecified queries. Feedback-based query reformulation approaches like pseudo-relevance feedback (PRF) alleviate this by incorporating additional evidence from retrieved or generated content. However, existing embedding-space approaches commonly aggregate or interpolate feedback vectors, implicitly treating them as complete target representations. Such pointwise fusion can unnecessarily suppress query corrections along directions not expressed by individual feedback items, even when these directions are supported by complementary evidence. To address this limitation, we propose Directional Posterior Feedback Fusion (DIFF), a query reformulation framework that models feedback as directional evidence rather than target points. DIFF represents feedback as confidence-weighted directional evidence in the tangent space of the original query and jointly infers a controlled query update through prior-regularized posterior fusion, without retraining the underlying dense retriever. Experiments across multiple dense retrievers and different feedback sources show that DIFF achieves the best overall retrieval performance, demonstrating the effectiveness of explicitly modeling how feedback should constrain query updates.

Wed 21 OctInformation Retrieval
The gist
Short or unclear search queries often make it hard for computers to find the right answers. The authors show a new method that improves these queries by using feedback differently. Instead of mixing feedback as single fixed points, their approach treats feedback as directions that guide how to change the query. This leads to better search results without needing to retrain the search system. Their tests show that this new way of using feedback works better than older methods across various search setups.
Open → 2610.29464v1

Scientific literature curation improved with verbal-action agent system

SciCurateBench: Curating Experimental Data from Heterogeneous Scientific Literature with Verbal-Action Agents

Abstract: Scientific research increasingly depends on curating experimental data from literature. Scientific data curation requires verifying exact experimental conditions (e.g., specific assays, clinical outcomes, and data availability) across long, heterogeneous files, which is a highly time-consuming process. Existing benchmarks evaluate only broad relevance on short queries, failing to capture this complexity. We introduce SciCurateBench, a benchmark of 200 expert-annotated papers from major life-science journals that challenges systems to verify conjunctive biological experimental criteria from raw, unprocessed scientific literature. We show that plain LLMs and popular reasoning, tool-using, and feedback-based agents all struggle with this task. To study what makes the task tractable, we further propose LangPoli, an integrative system in which a policy agent generates natural language action guidelines that actor agents convert into executable tools. LangPoli showed strong performance on both Paper Classification (83.16% F1 vs. 58.22% for its base LLM) and Structured Information Retrieval, and also improves re-ranking on the public benchmarks of LitSearch, BEIR, and TREC. We further explore curriculum transfer, a method that distills generalizable strategies from solved tasks to accelerate the solving of related problems. We hope this work facilitates research in truly practical scientific AI assistance. The anonymized pre-release can be found from https://anonymous.4open. science/r/scicuratebench-D885/.

Fri 9 OctComputation and Language
The gist
Collecting exact experimental data from scientific papers is very slow and difficult, especially because studies report many specific details spread out in long documents. The authors created a new test called SciCurateBench that checks how well computer systems can find very precise biological experiment details from real research papers. They found that popular language models and tools struggle with this task. To help, they designed a system called LangPoli, which uses agents that write natural language instructions that other agents turn into effective actions, improving accuracy significantly. They also explored ways to teach these systems to transfer learned strategies to similar tasks faster.
Open → 2610.14393v1

Open citation data improves author name disambiguation benchmarks

OC-AND: A Citation-Enriched Benchmark for Author Name Disambiguation

Abstract: Author Name Disambiguation benchmarks are predominantly derived from closed infrastructures that may not reflect conditions in open bibliographic environments. We present OC-AND, an open citation-enriched benchmark for AND constructed by remapping the WhoIsWho dataset with bibliographic and citation metadata from OpenCitations. The dataset is produced through a reproducible five-stage pipeline involving DOI verification, metadata validation, OpenCitations enrichment, and author-centric transformation. OC-AND retains the WhoIsWho ground-truth files and partition structure unchanged, while its publication files are restricted to publications that could be matched to a validated DOI and an OpenCitations record. It introduces realistic characteristics of open environments: heterogeneous metadata completeness, absent abstracts and affiliations, asymmetric citation coverage, and sparse graph connectivity. The dataset contains author records with associated citations, enabling evaluation of AND methods in settings closer to real-world open scholarly infrastructure scenarios. OC-AND is released as open data with complete provenance documentation and pipeline code, supporting reproducible research and the development of citation-aware disambiguation approaches.

Fri 9 OctDigital Libraries
The gist
It is hard to tell which authors wrote which papers when different authors have similar or identical names. The authors of this work created a new open dataset called OC-AND to test how well computer programs can solve this problem. They combined existing author name data with freely available citation information, making it more like the messy real world of open research data. This new dataset lets people test their author-matching methods under realistic conditions where some information is missing. The dataset and method to create it are fully open, so others can reproduce and build on their work.
Open → 2610.13462v1

Scholarly reading paths reveal personal curiosity and research styles

Feeling Wistful: Reflecting on Scholarly Sensibilities with Creative Reading Traces

Abstract: Researchers often read before they can articulate what they are looking for. As AI increasingly mediates scholarly search and synthesis, understanding and preserving the idiosyncratic judgments guiding early exploration become important. We call these evolving orientations scholarly sensibilities. To understand curiosity-driven reading, we first examined Wikipedia rabbitholing, a self-directed browsing practice, then designed Wistful, a research probe for open-ended scholarly exploration that captures reading paths as creative reading traces. We studied Wistful with 16 HCI researchers---eight junior and eight senior---in comparison with their usual workflows. Researchers approached the same scholarly landscape differently, with familiarity and personal interests shaping what they pursued and semantic proximity and scholarly links shaping their paths. Their traces made these differences visible and let readers revisit and compare their paths. We position creative reading traces as artifacts for reflection and exchange and as a means of studying how scholarly sensibilities are expressed through reading.

Wed 7 OctHuman-Computer Interaction
The gist
People often explore research topics before they know exactly what they want to find. The authors studied how researchers browse scholarly content in a personal and curious way. They created a tool called Wistful that records the unique routes researchers take through articles, showing how their interests and knowledge shape their reading. This helps make those reading choices visible and easier to reflect on or share with others. Such reading paths can help understand how scholars think when exploring ideas.
Open → 2610.10890v1

Combining retrieval methods improves Taiwanese history question answers

Bridging Static and Agentic RAG for Taiwanese Historical Question Answering

Abstract: Agentic retrieval-augmented generation (RAG) enables language models to adapt retrieval based on previously retrieved evidence, but it remains unclear whether such adaptive orchestration consistently outperforms well-designed static pipelines. We conduct a controlled comparison of agentic and static RAG for Taiwanese historical question answering, sharing the same generator and hybrid retrieval backend. Despite similar aggregate performance, the two pipelines differ on 70.83% of questions, with their advantages largely canceling out when averaged. An oracle that selects the better response per question improves the composite score by 0.2417 over the better individual pipeline, revealing substantial headroom for question-level selection. We therefore introduce a post-hoc selector that compares the two responses and their cited evidence, significantly outperforming either individual pipeline and recovering 60.34% of the oracle headroom. These results show that aggregate comparisons can obscure meaningful question-level differences between retrieval strategies, suggesting that exploiting their complementarity may be more fruitful than seeking a universally superior pipeline.

Sat 19 SeptComputation and LanguageInformation Retrieval
The gist
Answering history questions about Taiwan can be done using different methods to find helpful information. One method changes its search based on what it finds, while another uses a fixed plan. The authors found neither method is best for all questions; instead, they work best when combined and their strengths are chosen question by question. They built a system that picks the better answer each time, improving overall results. This shows that mixing search strategies and choosing between them can help answer questions more accurately.
Open → 2609.23056v1

Structural similarity improves cross language scientific sentence classification

Improving Cross-Lingual Transfer for Sequential Sentence Classification in Research Papers via Structural Similarity

Abstract: Sequential sentence classification (SSC) is an essential task for structuring scientific publications, and extending SSC research to languages other than English can improve accessibility to scientific knowledge in multilingual digital libraries. Cross-lingual transfer is a promising approach to address the scarcity of training data in non-English languages. Prior work on other natural language processing tasks has shown the benefits of capturing linguistic similarity between source and target languages. However, SSC inherently depends on patterns at the discourse level, such as label sequences and positional regularities, which appear consistently across languages regardless of linguistic differences. To examine the factors that determine transfer success in SSC, we constructed a multilingual SSC dataset covering 13 non-English languages collected from five academic databases. Our cross-lingual transfer experiments, using both encoder-based and generative models, show that linguistic proximity has no consistent predictive power for transfer performance, whereas structural similarity in rhetorical organization shows a weak but consistent positive correlation across models. After controlling for source-language performance, the similarity of label distributions is the most consistent predictor. Building on this finding, we propose a set of three methods that explicitly leverage structural information using generative models. In the in-domain evaluation, the best combination reaches parity with the strongest encoder baselines, and in transfer to languages unseen during training, it outperforms the strongest encoder baseline.

Thu 17 SeptComputation and LanguageDigital Libraries
The gist
Scientific papers often organize sentences into sections like introduction, methods, and results, which follow similar patterns across languages. This paper shows that using the structural similarities of these patterns helps computers better understand and classify sentences in scientific papers written in different languages. The authors created a multilingual dataset and found that language closeness matters less than how sentences are organized. They developed methods that use this structure information and improved classification accuracy when transferring models to languages without training data.
Open → 2609.19650v1

Graph structure cuts cost and improves answers in document-based question systems

When Is Graph Structure Worth Its Cost? The Case for Structure Pricing in Retrieval-Augmented Generation

Abstract: Graph-based retrieval-augmented generation (RAG) can help answer questions that require information from many documents. However, building a graph often requires many language-model calls during ingestion. It is therefore important to ask whether its quality gains justify the additional cost. We present EffiRAG, a graph-based RAG system designed to reduce this cost. It uses the graph to locate relevant passages and generates answers from the original text. This design preserves source information while keeping graph construction and query processing lightweight. We evaluate EffiRAG on UltraDomain, which contains 120 open-ended questions from four domains. Compared with LightRAG-hybrid, EffiRAG produces the preferred answer on 93 questions. LightRAG is preferred on 7, and the remaining 20 are splits. EffiRAG also reduces total system cost by 57 percent, from USD 0.952 to USD 0.408. The cost includes language-model calls during ingestion and querying. The advantage remains as the corpus grows. At 10 and 20 documents per domain, EffiRAG uses a lightweight, non-LLM filter to skip low-salience chunks. It remains preferred over LightRAG-hybrid. It costs 4.2 times and 4.5 times less, respectively. The comparisons identify different quality-cost trade-offs. Graph-based RAG systems should therefore be evaluated by both answer quality and cost. The results favor graph structure that locates and preserves source evidence.

Wed 16 SeptArtificial Intelligence
The gist
Answering questions that need info from many documents can be slow and costly if the system reads everything fully. The authors made EffiRAG, a method that uses a map (graph) of document parts to quickly find important text to answer questions. This saves money and keeps answers better by not losing the original info. Their tests show EffiRAG gives better answers more often than a similar system while spending less than half the money on language model calls.
Open → 2609.18099v1