| Challenge: | Existing evidence-based factchecking datasets contain synthetic claims and lack real-world verification. |
| Approach: | They propose a dataset for evidence-based fact-checking of health-related claims that evaluates their truthfulness against scientific articles. |
| Outcome: | The proposed dataset evaluates real-world claims against scientific articles. |
Similar Papers
HealthFC: Verifying Health Claims with Evidence-Based Medical Fact-Checking (2024.lrec-main)
Copied to clipboard
| Challenge: | determining the trustworthiness of online medical content is challenging in the digital age . fact-checking is an approach to assess the veracity of factual claims . a new dataset is presented to help advance automated fact- checking . |
| Approach: | They propose a dataset that assesses the veracity of factual claims using evidence from credible sources. |
| Outcome: | The proposed dataset can be used for automated fact-checking tasks. |
Explainable Automated Fact-Checking for Public Health Claims (2020.emnlp-main)
Copied to clipboard
| Challenge: | a few blind spots exist in the state-of-the-art in fact-checking for political claims. |
| Approach: | They propose to use a dataset of 11.8K claims to explain fact-check labels for claims . they define and evaluate three coherence properties of explanation quality with humans . |
| Outcome: | The proposed model can be trained on in-domain data and evaluates its coherence properties with humans and computationally. |
Comparing Knowledge Sources for Open-Domain Scientific Claim Verification (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing systems for fact-checking scientific claims assume that the documents containing the evidence are already provided and annotated or contained in a limited corpus. |
| Approach: | They perform an array of experiments to test the performance of open-domain claim verification systems on four datasets of biomedical and health claims in different settings. |
| Outcome: | The proposed system performs better with biomedical and health claims, while Wikipedia is more suited for everyday health concerns. |
PubHealthTab: A Public Health Table-based Dataset for Evidence-based Fact Checking (2022.findings-naacl)
Copied to clipboard
| Challenge: | Fact-checking is the task of establishing the veracity of factual information, commonly performed manually by journalists. |
| Approach: | They propose a table fact-checking dataset based on real world public health claims and noisy evidence tables from sources similar to those used by fact checkers. |
| Outcome: | The proposed dataset achieves an overall F1 score of 0.73 . |
Check-COVID: Fact-Checking COVID-19 News Claims with Scientific Evidence (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing fact-checking benchmarks require systems to verify claims from everyday text against evidence from scientific journal articles. |
| Approach: | They propose a benchmark system that checks claims from news against scientific journal articles and veracity labels. |
| Outcome: | The new benchmark achieves F1 scores of 76.99 and 69.90 on both a fact-checking specific system and GPT-3.5, respectively. |
Automated Fact-Checking of Claims from Wikipedia (2020.lrec-1)
Copied to clipboard
| Challenge: | Fact checking datasets such as FEVER and SNLI suffer from limited applicability due to synthetic nature of claims and/or evidence written by annotators that differ from real claims and evidence on the internet. |
| Approach: | They present a dataset of 124k+ triples consisting of a claim, context and an evidence document extracted from English Wikipedia articles and citations. |
| Outcome: | The proposed dataset is the largest fact checking dataset consisting of real claims and evidence to date. |
MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claims (D19-1)
Copied to clipboard
Isabelle Augenstein, Christina Lioma, Dongsheng Wang, Lucas Chaves Lima, Casper Hansen, Christian Hansen, Jakob Grue Simonsen
| Challenge: | Existing efforts to verify factual claims are limited by small datasets or artificially constructed datasets. |
| Approach: | They propose to use the largest publicly available dataset of naturally occurring factual claims for automatic claim verification. |
| Outcome: | The proposed model outperforms baseline models and evidence pages significantly. |
MedFact: A Large-scale Chinese Dataset for Evidence-based Medical Fact-checking of LLM Responses (2025.emnlp-main)
Copied to clipboard
Tong Chen, Zimu Wang, Yiyi Miao, Haoran Luo, Sun Yuanfei, Wei Wang, Zhengyong Jiang, Procheta Sen, Jionglong Su
| Challenge: | Existing medical fact-checking datasets focus on human-generated content, leaving the verification of content generated by large language models (LLMs) relatively unexplored. |
| Approach: | They propose to use Chinese medical fact-checking datasets to verify LLM-generated medical content by combining in-context learning and fine-tuning. |
| Outcome: | The first evidence-based Chinese medical fact-checking dataset of LLM-generated medical content consists of 1,321 questions and 7,409 claims . |
Step-by-Step Fact Verification System for Medical Claims with Explainable Reasoning (2025.naacl-short)
Copied to clipboard
| Challenge: | Fact verification (FV) aims to assess the veracity of a claim based on relevant evidence. |
| Approach: | They propose to use iterative fact verification to assess the veracity of a claim based on relevant evidence. |
| Outcome: | The proposed system improves on three medical fact-checking datasets and evaluates with multiple settings including different LLMs, external web search, and structured reasoning using logic predicates. |
Fact or Fiction: Verifying Scientific Claims (2020.emnlp-main)
Copied to clipboard
David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, Hannaneh Hajishirzi
| Challenge: | SciFact is a dataset of 1.4K expert-written scientific claims paired with evidence-containing abstracts annotated with labels and rationales. |
| Approach: | They construct a dataset of 1.4K scientific claims paired with evidence-containing abstracts annotated with labels and rationales to test their system. |
| Outcome: | The proposed system can verify claims related to COVID-19 by identifying evidence from the CORD-19 corpus. |