SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables (2023.emnlp-main)
Copied to clipboard
| Challenge: | Current scientific fact-checking benchmarks exhibit several shortcomings, such as biases arising from crowd-sourced claims and an over-reliance on text-based evidence. |
| Approach: | They present a dataset of 1.2K expert-verified scientific claims that require compositional reasoning for verification. |
| Outcome: | The proposed model outperforms existing models in table-based pretraining models and large language models. |
Similar Papers
Table-Text Alignment: Explaining Claim Verification Against Tables in Scientific Papers (2025.findings-emnlp)
Copied to clipboard
| Challenge: | predicting the final label alone is insufficient and offers limited interpretability. |
| Approach: | They propose to reframe table–text alignment as an explanation task requiring models to identify the table cells essential for claim verification. |
| Outcome: | The proposed taxonomy improves claim verification performance and most LLMs fail to recover human-aligned rationales, suggesting that their predictions do not stem from faithful reasoning. |
SciFact-Open: Towards open-domain scientific claim verification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Current scientific claim verification systems can achieve very strong performance on limited contexts, in some cases approaching human agreement. |
| Approach: | They propose to pool and annotate top predictions from four state-of-the-art scientific claim verification models to evaluate their performance against large corpora. |
| Outcome: | The proposed system performs well on a corpus of 500K scientific abstracts. |
Fact or Fiction: Verifying Scientific Claims (2020.emnlp-main)
Copied to clipboard
David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, Hannaneh Hajishirzi
| Challenge: | SciFact is a dataset of 1.4K expert-written scientific claims paired with evidence-containing abstracts annotated with labels and rationales. |
| Approach: | They construct a dataset of 1.4K scientific claims paired with evidence-containing abstracts annotated with labels and rationales to test their system. |
| Outcome: | The proposed system can verify claims related to COVID-19 by identifying evidence from the CORD-19 corpus. |
ClaimDB: A Fact Verification Benchmark over Large Structured Data (2026.acl-long)
Copied to clipboard
| Challenge: | despite substantial progress in fact-verification benchmarks, this setting remains largely underexplored. |
| Approach: | They propose a fact-verification benchmark where evidence for claims is derived from compositions of millions of records and multiple tables. |
| Outcome: | The proposed benchmarks score below 55% accuracy with 30 state-of-the-art LLMs and are released on github. |
Pushing the Frontiers of Scientific Fact-Checking: The SCINLP Dataset (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly being used to understand how scientific research evolves, drawing growing interest from the research community. |
| Approach: | They propose a scientific fact-checking dataset, SCINLP, tailored to the NLP domain that verifies the veracity of scientific research questions across varying rationale contexts. |
| Outcome: | The proposed framework examines scientific claims and research focus from a curated collection of influential and reputable NLP papers published between 2000 and 2024. |
ClimateViz: A Benchmark for Statistical Reasoning and Fact Verification on Scientific Charts (2025.emnlp-main)
Copied to clipboard
| Challenge: | Scientific fact-checking has largely focused on textual and tabular sources, neglecting scientific charts. |
| Approach: | They propose a benchmark for scientific fact-checking grounded in scientific charts . climateViz comprises 49,862 claims paired with 2,896 visualizations . results show current models struggle to perform fact- checking when statistical reasoning is required . |
| Outcome: | The climateviz benchmark is the first large-scale benchmark for scientific fact-checking . it includes 49,862 claims paired with 2,896 visualizations labeled as support, refute, or not enough . |
How Robust Are the QA Models for Hybrid Scientific Tabular Data? A Study Using Customized Dataset (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing tabular QA models are lacking in understanding their robustness on scientific information. |
| Approach: | They propose a dataset to assess the robustness of tabular QA models on scientific hybrid tabular data. |
| Outcome: | The proposed model performs well on scientific tables and text, while the best score is 0.462. |
SciTrue: Evidence-Grounded Claim Verification in Science (2026.eacl-demo)
Copied to clipboard
| Challenge: | Existing systems often exhibit unverifiable attributions, shallow evidence mapping, and hallucinated citations. |
| Approach: | They propose a claim verification system that provides source-level accountability and evidence traceability. |
| Outcome: | SciTrue outperforms RAG-based baselines in summary traceability, attribution accuracy, and context alignment in a human evaluation of 300 attributions. |
CLAIMCHECK: How Grounded are LLM Critiques of Scientific Papers? (2025.findings-emnlp)
Copied to clipboard
Jiefu Ou, William Gantt Walden, Kate Sanders, Zhengping Jiang, Kaiser Sun, Jeffrey Cheng, William Jurayj, Miriam Wanner, Shaobo Liang, Candice Morgan, Seunghoon Han, Weiqi Wang, Chandler May, Hannah Recknor, Daniel Khashabi, Benjamin Van Durme
| Challenge: | CLAIMCHECK is an annotated dataset of NeurIPS 2023 and 2024 submissions and reviews from OpenReview. |
| Approach: | They annotate NeurIPS 2023 and 2024 submissions and reviews for weaknesses and dispute them for fine-grained labels of validity, objectivity, and type of the identified weaknesses. |
| Outcome: | The proposed dataset is richly annotated by ML experts for weaknesses statements in the reviews and the claims that they dispute, as well as fine-grained labels of validity, objectivity, and type of the identified weaknesses. |
SciVer: Evaluating Foundation Models for Multimodal Scientific Claim Verification (2025.acl-long)
Copied to clipboard
| Challenge: | Existing scientific claim verification benchmarks focus on textual content alone or on verifying claims based on a single table. |
| Approach: | They propose to use SciVer to evaluate the ability of foundation models to verify claims within a multimodal scientific context. |
| Outcome: | The proposed model outperforms 21 state-of-the-art models and human experts on SciVer. |