Check-COVID: Fact-Checking COVID-19 News Claims with Scientific Evidence (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing fact-checking benchmarks require systems to verify claims from everyday text against evidence from scientific journal articles. |
| Approach: | They propose a benchmark system that checks claims from news against scientific journal articles and veracity labels. |
| Outcome: | The new benchmark achieves F1 scores of 76.99 and 69.90 on both a fact-checking specific system and GPT-3.5, respectively. |
Similar Papers
COVID-Fact: Fact Extraction and Verification of Real-World Claims on COVID-19 Pandemic (2021.acl-long)
Copied to clipboard
| Challenge: | a new method for fact-checking is needed to detect disinformation on the web . a dataset COVID-Fact contains 4,086 claims concerning the COVId-19 pandemic . |
| Approach: | They propose a FEVER-like dataset COVID-Fact of 4,086 claims concerning the COVId-19 pandemic . they automatically detect true claims and their source articles and generate counter-claims using automatic methods . |
| Outcome: | The proposed method reduces the cost of building domain-specific datasets for detecting misinformation . the proposed dataset contains 4,086 claims concerning the COVID-19 pandemic . |
Fact or Fiction: Verifying Scientific Claims (2020.emnlp-main)
Copied to clipboard
David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, Hannaneh Hajishirzi
| Challenge: | SciFact is a dataset of 1.4K expert-written scientific claims paired with evidence-containing abstracts annotated with labels and rationales. |
| Approach: | They construct a dataset of 1.4K scientific claims paired with evidence-containing abstracts annotated with labels and rationales to test their system. |
| Outcome: | The proposed system can verify claims related to COVID-19 by identifying evidence from the CORD-19 corpus. |
Adapting Open Domain Fact Extraction and Verification to COVID-FACT through In-Domain Language Modeling (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to verify scientifically false online information are limited by the lack of training data in the scientific domain. |
| Approach: | They propose an in-domain language modeling method for fact extraction and verification systems . they use SCIFACT to extract scientifically false online information . |
| Outcome: | The proposed method improves accuracy 30% on SCIFACT dataset . state-of-the-art model achieves only 46.6% precision, which is hard to be trusted for users. |
Fighting the COVID-19 Infodemic: Modeling the Perspective of Journalists, Fact-Checkers, Social Media Platforms, Policy Makers, and the Society (2021.findings-emnlp)
Copied to clipboard
Firoj Alam, Shaden Shaar, Fahim Dalvi, Hassan Sajjad, Alex Nikolov, Hamdy Mubarak, Giovanni Da San Martino, Ahmed Abdelali, Nadir Durrani, Kareem Darwish, Abdulaziz Al-Homaid, Wajdi Zaghouani, Tommaso Caselli, Gijs Danoe, Friso Stolk, Britt Bruntink, Preslav Nakov
| Challenge: | a dataset of 16K manually annotated tweets is used to analyze disinformation . the democratic nature of social media has raised questions about the quality and the factuality of the information that is shared on these platforms. |
| Approach: | They use a dataset of manually annotated tweets to analyze COVID-19 disinformation . they show that tweets contain fake cures, rumors, conspiracy theories and xenophobia . |
| Outcome: | The proposed dataset shows that it is useful in monolingual vs. multilingual settings. |
Scientific Fact-Checking: A Survey of Resources and Approaches (2023.findings-acl)
Copied to clipboard
| Challenge: | Fact-checking is the task of assessing the veracity of factual claims based on credible evidence and background knowledge. |
| Approach: | They propose to automate scientific fact-checking using natural language processing to assess the veracity of factual claims based on credible evidence and background knowledge. |
| Outcome: | The proposed methods can help combat the spread of misinformation and help individuals understand new scientific breakthroughs. |
Evidence-based Fact-Checking of Health-related Claims (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evidence-based factchecking datasets contain synthetic claims and lack real-world verification. |
| Approach: | They propose a dataset for evidence-based fact-checking of health-related claims that evaluates their truthfulness against scientific articles. |
| Outcome: | The proposed dataset evaluates real-world claims against scientific articles. |
PANACEA: An Automated Misinformation Detection System on COVID-19 (2023.eacl-demo)
Copied to clipboard
Runcong Zhao, Miguel Arana-catania, Lixing Zhu, Elena Kochkina, Lin Gui, Arkaitz Zubiaga, Rob Procter, Maria Liakata, Yulan He
| Challenge: | Using social media and fact-checking to detect misinformation is not enough to prevent the spread of false information. |
| Approach: | They propose a web-based misinformation detection system PANACEA which has two modules, fact-checking and rumour detection. |
| Outcome: | The system outperforms state-of-the-art methods and adapts graph convolutional networks model to detect rumours based on tweets rather than knowledge bases. |
Explainable Automated Fact-Checking for Public Health Claims (2020.emnlp-main)
Copied to clipboard
| Challenge: | a few blind spots exist in the state-of-the-art in fact-checking for political claims. |
| Approach: | They propose to use a dataset of 11.8K claims to explain fact-check labels for claims . they define and evaluate three coherence properties of explanation quality with humans . |
| Outcome: | The proposed model can be trained on in-domain data and evaluates its coherence properties with humans and computationally. |
CoVERT: A Corpus of Fact-checked Biomedical COVID-19 Tweets (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing fact-checking resources cover COVID-19 related information in news, but there is no dataset providing fact- checked COVId-19 related tweets with detailed annotations for biomedical entities, relations and relevant evidence. |
| Approach: | They propose a fact-checked corpus of tweets with annotations for biomedical entities, relations and relevant evidence for COVID-19 related tweets. |
| Outcome: | The proposed dataset provides fact-checked COVID-19 related tweets with detailed annotations for biomedical entities, relations and relevant evidence. |
Automated Fact-Checking of Claims from Wikipedia (2020.lrec-1)
Copied to clipboard
| Challenge: | Fact checking datasets such as FEVER and SNLI suffer from limited applicability due to synthetic nature of claims and/or evidence written by annotators that differ from real claims and evidence on the internet. |
| Approach: | They present a dataset of 124k+ triples consisting of a claim, context and an evidence document extracted from English Wikipedia articles and citations. |
| Outcome: | The proposed dataset is the largest fact checking dataset consisting of real claims and evidence to date. |