| Challenge: | NSF-SciFy contains 2.8 million claims from 400,000 abstracts spanning all science and mathematics disciplines. |
| Approach: | They propose to use a dataset to extract scientific claims from National Science Foundation award abstracts and to use it to refine language models. |
| Outcome: | The proposed method improves non-technical abstract generation, claim extraction, and investigation proposal extraction tasks while maintaining high precision and lower recall. |
Similar Papers
SciFact-Open: Towards open-domain scientific claim verification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Current scientific claim verification systems can achieve very strong performance on limited contexts, in some cases approaching human agreement. |
| Approach: | They propose to pool and annotate top predictions from four state-of-the-art scientific claim verification models to evaluate their performance against large corpora. |
| Outcome: | The proposed system performs well on a corpus of 500K scientific abstracts. |
SciER: An Entity and Relation Extraction Dataset for Datasets, Methods, and Tasks in Scientific Documents (2024.emnlp-main)
Copied to clipboard
| Challenge: | Scientific information extraction (SciIE) is critical for converting unstructured knowledge from scholarly articles into structured data. |
| Approach: | They propose to use a scientific entity and relation extraction dataset to capture interactions between entities in full texts. |
| Outcome: | The proposed dataset captures the intricate use and interactions among entities in full texts and provides an out-of-distribution test set to offer a more realistic evaluation. |
SciTrue: Evidence-Grounded Claim Verification in Science (2026.eacl-demo)
Copied to clipboard
| Challenge: | Existing systems often exhibit unverifiable attributions, shallow evidence mapping, and hallucinated citations. |
| Approach: | They propose a claim verification system that provides source-level accountability and evidence traceability. |
| Outcome: | SciTrue outperforms RAG-based baselines in summary traceability, attribution accuracy, and context alignment in a human evaluation of 300 attributions. |
SciDMT: A Large-Scale Corpus for Detecting Scientific Mentions (2024.lrec-main)
Copied to clipboard
| Challenge: | SciDMT is an enhanced and expanded corpus for scientific mention detection . existing corpora are limited by their small volume and entity linking capabilities . |
| Approach: | They propose to enhance SciDMT, an annotated scientific corpus for scientific mention detection. |
| Outcome: | The proposed corpus is the largest for scientific entity mention detection . it is based on deep learning architectures like SciBERT and GPT-3.5 . |
SciImpact: A Multi-Dimensional, Multi-Field Benchmark for Scientific Impact Prediction (2026.findings-acl)
Copied to clipboard
| Challenge: | Prior work on scientific impact prediction has focused on citation counts and its variants, leaving limited evaluation of models’ capability to reason about other dimensions. |
| Approach: | They propose a large-scale, multi-dimensional benchmark for scientific impact prediction spanning 19 fields. |
| Outcome: | The proposed model outperforms larger models and close-source models in a wide range of fields and measures of scientific impact across 19 fields. |
Pushing the Frontiers of Scientific Fact-Checking: The SCINLP Dataset (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly being used to understand how scientific research evolves, drawing growing interest from the research community. |
| Approach: | They propose a scientific fact-checking dataset, SCINLP, tailored to the NLP domain that verifies the veracity of scientific research questions across varying rationale contexts. |
| Outcome: | The proposed framework examines scientific claims and research focus from a curated collection of influential and reputable NLP papers published between 2000 and 2024. |
SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables (2023.emnlp-main)
Copied to clipboard
| Challenge: | Current scientific fact-checking benchmarks exhibit several shortcomings, such as biases arising from crowd-sourced claims and an over-reliance on text-based evidence. |
| Approach: | They present a dataset of 1.2K expert-verified scientific claims that require compositional reasoning for verification. |
| Outcome: | The proposed model outperforms existing models in table-based pretraining models and large language models. |
Fact or Fiction: Verifying Scientific Claims (2020.emnlp-main)
Copied to clipboard
David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, Hannaneh Hajishirzi
| Challenge: | SciFact is a dataset of 1.4K expert-written scientific claims paired with evidence-containing abstracts annotated with labels and rationales. |
| Approach: | They construct a dataset of 1.4K scientific claims paired with evidence-containing abstracts annotated with labels and rationales to test their system. |
| Outcome: | The proposed system can verify claims related to COVID-19 by identifying evidence from the CORD-19 corpus. |
CitationIE: Leveraging the Citation Graph for Scientific Information Extraction (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work on scientific information extraction (SciIE) considers extraction solely based on the content of an individual paper, without considering the paper’s place in the broader literature. |
| Approach: | They propose to automate the extraction of key information from scientific documents by leveraging a complementary source: the citation graph of referential links between citing and cited papers. |
| Outcome: | The proposed model improves on a set of English-language scientific documents. |
POLYIE: A Dataset of Information Extraction from Polymer Material Scientific Literature (2024.naacl-long)
Copied to clipboard
Jerry Cheung, Yuchen Zhuang, Yinghao Li, Pranav Shetty, Wantian Zhao, Sanjeev Grampurohit, Rampi Ramprasad, Chao Zhang
| Challenge: | SciIE datasets for polymer materials are lacking for this class of materials . POLYIE is curated from 146 full-length polymer scholarly articles . |
| Approach: | They propose a SciIE dataset for polymer materials that uses entity annotations from 146 full-length articles. |
| Outcome: | The proposed dataset is curated from 146 full-length polymer scholarly articles . it presents challenges due to diverse lexical formats of entities and ambiguity between entities . |