Papers for

scientific ai developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Scientific language models improve accuracy using mechanism knowledge structure

From Papers to Mechanisms: An Evidence-Grounded Knowledge Substrate for Scientific Language Models

Abstract: Scientific language models often access literature through untyped text chunks, which fragment the functional and evidential structure required for mechanism-rich questions. We introduce an evidence-grounded mechanism knowledge substrate that organizes scientific literature into provenance-linked evidence units, role-typed entities, and directed mechanism paths. We instantiate it as MS$^3$, a Material-Sensor-Signal-System schema for conductive-fiber flexible sensors, over 13,689 papers, 131,083 evidence items, and 26,648 mechanism objects. On in-domain and coverage-shift question-answering benchmarks, we compare closed-book generation, Web search, Raw-PDF RAG, and MS$^3$ retrieval across ten language models. MS$^3$ improves macro-averaged scientific correctness. It also improves citation entailment and answer completeness. These results support mechanism substrates as a reliable representation layer for scientific language models and motivate a source-repair workflow in which insufficient MS$^3$ evidence triggers targeted retrieval from its linked papers rather than assuming that a user has already supplied the correct PDFs.

Mon 5 OctArtificial Intelligence
The gist
Scientific papers contain complex information about how things work, but current language models read them as plain text chunks, missing important connections. The authors created a special organized system, called MS$^3$, that breaks down scientific papers into detailed evidence units, entities, and linked paths showing mechanisms. They tested MS$^3$ on thousands of papers and found it helps language models answer questions more accurately and completely. This approach encourages checking the original papers only when needed instead of assuming all information is perfect upfront.
Open → 2610.06248v1

Multimodal agents struggle to fully use scientific evidence for reasoning

Sci-MMR: Benchmarking Multi-Step Evidence-Grounded Scientific Reasoning in Multimodal Agents

Abstract: Autonomous research agents are increasingly expected to search the literature, analyze experimental evidence, and generate scientific hypotheses. These capabilities require multi-step evidence grounded reasoning that progressively acquires, integrates, and verifies evidence before reaching a conclusion. Existing multimodal benchmarks, however, largely evaluate final-answer accuracy, leaving open whether predictions are actually supported by traceable scientific evidence. We introduce Sci-MMR, a benchmark for multi-step evidence-grounded scientific reasoning built on structured argument graphs linking scientific claims, citation-grounded knowledge, visual evidence, and supporting regions. Sci-MMR comprises 235 multi-hop reasoning tasks spanning four scientific disciplines, with an average of nine figure panels per task. Evaluating eight frontier multimodal models, we find that answer accuracy consistently exceeds complete-evidence recovery rate by more than 20%, revealing a substantial gap that answer-only evaluation is structurally unable to capture. Through controlled interventions, we identify two fundamental bottlenecks. First, evidence acquisition: models struggle to extract complete structured evidence from scientific figures, accounting for 57.2% of failures. While cropping tools yield modest gains (+4.5 points), providing gold evidence improves accuracy by up to 37.0 points, indicating difficulty in assembling complete multi-region evidence. Second, evidence integration: models struggle to translate available evidence into correct conclusions, accounting for 31.8% of failures, while even with gold evidence the strongest model achieves only 69.1% accuracy on the hardest tasks. These findings indicate that current answer-centric benchmarks substantially overestimate the evidence-grounded reasoning capabilities of multimodal research agents

Thu 10 SeptArtificial Intelligence
The gist
Many multimodal AI systems can give correct scientific answers but often can't show all the evidence that supports those answers. The authors created Sci-MMR, a test that checks if AI models can find, combine, and explain multiple pieces of scientific evidence from text and images. They found that current AI models miss a lot of the evidence, especially from scientific figures, and have trouble using the evidence correctly to reach the right conclusions. This means current tests might overestimate how well AI can reason based on real scientific evidence.
Open → 2609.11243v1