Tal Schuster, Darsh Shah, Yun Jie Serene Yeo, Daniel Roberto Filizzola Ortiz, Enrico Santus, Regina Barzilay
| Challenge: | Prior research has shown that data collection methods that use crowdsourcing introduce idiosyncratic biases that impact performance in unexpected ways. |
| Approach: | They propose a method to regularize the training data to avoid idiosyncrasies in the datasets that are used for fact verification. |
| Outcome: | The proposed model outperforms the existing model on the FEVER dataset, achieving 61.7% of the baseline. |
Similar Papers
Constrained Fact Verification for FEVER (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for fact verification rely on extracted evidence, but there is little work on understanding the reasoning process. |
| Approach: | They propose a method that enforces a closed-world reliance on extracted evidence to verify a claim's factuality. |
| Outcome: | The proposed model outperforms existing models on the FEVER shared task and shows that it is more accurate than previous models. |
FEVER: a Large-scale Dataset for Fact Extraction and VERification (N18-1)
Copied to clipboard
| Challenge: | 185,445 claims generated by altering sentences from Wikipedia are verified without knowledge of the sentence they were derived from. |
| Approach: | They propose a publicly available dataset for verification against textual sources, FEVER: Fact Extraction and VERification. |
| Outcome: | The proposed dataset achieves 31.87% accuracy on labeling a claim accompanied by the correct evidence, compared to 50.91% if we ignore the evidence. |
Counterfactual Debiasing for Fact Verification (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for debiasing factchecking models learn such biases instead of understanding the semantic relationship between the claim and evidence. |
| Approach: | They propose a counterfactual framework CLEVER which is augmentation-free and mitigates biases on the inference stage. |
| Outcome: | The proposed method is augmentation-free and mitigates biases on the inference stage. |
Exploring Listwise Evidence Reasoning with T5 for Fact Verification (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for fact verification use pretrained sequence-to-sequence transformers for sentence selection and label prediction. |
| Approach: | They propose a framework for fact verification that leverages pretrained sequence-to-sequence transformer models for sentence selection and label prediction. |
| Outcome: | The proposed framework scores higher than the second place approach on the blind test set . the proposed framework can be useful for a broader range of NLP tasks, the authors say . |
DeSePtion: Dual Sequence Prediction and Adversarial Examples for Improved Fact-Checking (2020.acl-main)
Copied to clipboard
Christopher Hidey, Tuhin Chakrabarty, Tariq Alhindi, Siddharth Varia, Kriste Krstovski, Mona Diab, Smaranda Muresan
| Challenge: | Fact Extraction and Verification datasets provide a resource for end-to-end fact-checking, requiring retrieval of evidence from Wikipedia to validate a veracity prediction. |
| Approach: | They propose a system that is resilient to attacks by multiple propositions, temporal reasoning, ambiguity and lexical variation and a sequence of evidence sentences and veracity relation predictions. |
| Outcome: | The proposed system is resilient to three realistic “attacks” and obtains state-of-the-art results due to improved evidence retrieval. |
Zero-shot Fact Verification by Claim Generation (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for fact verification require large datasets, which can be expensive. |
| Approach: | They propose a framework for training a robust fact verification model by using automatically generated claims that can be supported, refuted, or unverifiable from evidence from Wikipedia. |
| Outcome: | The proposed framework reduces the demand for human-annotated training data and improves a model's F1 from 50% to 77%, equivalent in performance to 2K+ manually-curated examples. |
Claim-Dissector: An Interpretable Fact-Checking System with Joint Re-ranking and Veracity Prediction (2023.findings-acl)
Copied to clipboard
| Challenge: | a novel latent variable model for fact-checking and analysis learns to identify the veracity of a claim and its relevant evidences. |
| Approach: | They propose to disentangle the per-evidence relevance probability and its contribution to the final veracity probability in an interpretable way. |
| Outcome: | The proposed model can achieve competitive results on the FEVER dataset while using significantly fewer parameters. |
Adversarial Removal of Demographic Attributes Revisited (D19-1)
Copied to clipboard
| Challenge: | Several approaches have been proposed to learn classifiers that are invariant (unbiased with respect) to protected attributes. |
| Approach: | They propose to use a diagnostic classifier trained on a held-out subsample to find protected attributes for mention detection at above-chance levels. |
| Outcome: | The proposed classifier generalizes poorly to new in-domain and new domains, suggesting it relies on correlations specific to their particular data sample. |
Claim Check-Worthiness Detection as Positive Unlabelled Learning (2020.findings-emnlp)
Copied to clipboard
| Challenge: | a unified approach to claim check-worthiness detection is a critical component of fact checking systems. |
| Approach: | They propose a unified approach which corrects for misinformation by positive unlabelled learning . they propose citation needed detection from Wikipedia and a ranking task which is a critical component of automatic fact checking systems. |
| Outcome: | The proposed method outperforms the state of the art in two of the three tasks studied in English. |
EX-FEVER: A Dataset for Multi-hop Explainable Fact Verification (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on fact verification lack a high-quality dataset for explainability . existing systems lack evidence retrieval and veracity prediction, limiting the ability to verify a claim . |
| Approach: | They propose a dataset for multi-hop explainable fact verification that summarises and modifies Wikipedia documents. |
| Outcome: | The proposed dataset aims to improve the accuracy of multi-hop explainable fact verification systems. |