Dataset links reviewer comments to paper evidence for better grounding
ReGround: Grounding Reviewer Comments in Multimodal Evidence
Computation and LanguageInformation Retrieval
Summary
Sometimes reviewers of scientific papers give comments that point to specific parts of the paper, but it's very hard to find exactly where in the paper those comments relate. The authors created a large collection of examples where reviewer comments are linked to the evidence in the paper, using author replies to help find the exact parts. They tested many ways to find these links and found it is a hard problem, especially because papers often have text, figures, and tables that all give clues. This new dataset helps computers learn to better understand and connect reviewer comments to the right parts of scientific papers.
What this means in practice
- •For scientific conference organizers: Improve automated tools that assist reviewers by linking their comments directly to the relevant parts of paper submissions for easier evaluation.
- •For academic paper management platforms: Develop features to highlight and track how reviewer comments relate to specific evidence and content types across text and figures in submissions.$Commercial implications: Enables product features that streamline peer review management and improve author-reviewer communication in submission platforms.
Authors
Serwar Basch, Lizhen Qu, Iryna Gurevych
Abstract
Reviewer comments naturally relate to specific parts of the reviewed paper, yet grounding these comments to the underlying evidence is difficult due to long multimodal documents. Existing benchmarks do not capture this setting and largely focus on explicit, information-seeking queries. We introduce ReGround, a large-scale dataset for reviewer comment grounding that links 10,267 reviewer comments to 16,274 evidence in the original anonymous submission of 3,656 papers. We build on a simple observation: author rebuttals often include explicit references to content of the submission used to address reviewer comments, providing a high-precision annotation source. We cast grounding as a retrieval task and evaluate a wide range of retrieval methods. Results show that retrieval over the entire paper content performs poorly, evidence-type inference is a major bottleneck, and multimodal evidence provides complementary signals that text alone misses. Our dataset exposes grounding reviewer comments as a difficult and practically important problem for scientific document understanding.