| Challenge: | Recent fact verification systems rely on neural network classifiers for veracity prediction, which lack explainability. |
| Approach: | They propose a model that generates natural logic-based inferences as proofs using lexical mutations between spans in the claim and the evidence retrieved. |
| Outcome: | The proposed model has highest label accuracy and second best score in the FEVER leaderboard. |
Similar Papers
QA-NatVer: Question Answering for Natural Logic-based Fact Verification (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has focused on natural logic, which operates directly on natural language by capturing the semantic relation of spans between an aligned claim and its evidence via set-theoretic operators. |
| Approach: | They propose to use question answering to predict natural logic operators using generalization capabilities of instruction-tuned language models. |
| Outcome: | The proposed approach outperforms the best baseline on a Danish verification dataset by 4.3 accuracy points. |
PRover: Proof Generation for Interpretable Reasoning over Rules (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that transformers can act as “soft theorem provers” by answering questions over explicitly provided knowledge in natural language. |
| Approach: | They propose a transformer-based model that answers binary questions over rule-bases and generates the corresponding proofs. |
| Outcome: | The proposed model generates proofs with an accuracy of 87% while maintaining or improving performance on the QA task. |
Automated Justification Production for Claim Veracity in Fact Checking: A Survey on Architectures and Approaches (2024.acl-long)
Copied to clipboard
| Challenge: | Current research focuses on predicting claim veracity through metadata analysis and language scrutiny, with an emphasis on justifying verdicts. |
| Approach: | They propose a comprehensive taxonomy for categorizing works based on various criteria and propose scalable methodologies for improving fact-checking explainability. |
| Outcome: | The proposed taxonomy identifies challenges while proposing future directions in fact-checking explainability. |
Exploring Listwise Evidence Reasoning with T5 for Fact Verification (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for fact verification use pretrained sequence-to-sequence transformers for sentence selection and label prediction. |
| Approach: | They propose a framework for fact verification that leverages pretrained sequence-to-sequence transformer models for sentence selection and label prediction. |
| Outcome: | The proposed framework scores higher than the second place approach on the blind test set . the proposed framework can be useful for a broader range of NLP tasks, the authors say . |
FaiRR: Faithful and Robust Deductive Reasoning over Natural Language (2022.acl-long)
Copied to clipboard
| Challenge: | Currently, black-box models generate both the proof graph and intermediate inferences within the same model and thus may be unfaithful. |
| Approach: | They propose a transformer-based model that can perform deductive reasoning on a logical rulebase containing rules and statements written in natural language. |
| Outcome: | The proposed model is robust to language perturbations and faster at inference than previous models on existing reasoning datasets. |
Constrained Fact Verification for FEVER (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for fact verification rely on extracted evidence, but there is little work on understanding the reasoning process. |
| Approach: | They propose a method that enforces a closed-world reliance on extracted evidence to verify a claim's factuality. |
| Outcome: | The proposed model outperforms existing models on the FEVER shared task and shows that it is more accurate than previous models. |
ClaimVer: Explainable Claim-Level Verification and Evidence Attribution of Text Through Knowledge Graphs (2024.findings-emnlp)
Copied to clipboard
Preetam Prabhu Srikar Dammu, Himanshu Naidu, Mouly Dewan, YoungMin Kim, Tanya Roosta, Aman Chadha, Chirag Shah
| Challenge: | Despite the fact that many fact-checking tools lack granularity and explainability, they lack the ability to be useful in various contexts. |
| Approach: | They propose a text validation framework that provides granular explanations for each claim and localizes the specific problematic content to reduce cognitive load. |
| Outcome: | The proposed framework provides granular explanations for each claim prediction and localizes and educates users on the specific content. |
Explainable Claim Verification via Knowledge-Grounded Reasoning with Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing claims verification models rely on annotated data, which is expensive to create at a large scale. |
| Approach: | They propose a model that can verify complex claims without annotated data . they leverage the in-context learning ability of Large Language Models to translate a claim into a First-Order-Logic clause . |
| Outcome: | The proposed model outperforms baseline models on three datasets . it performs well on the datasets, and the results are published online. |
Generating Fact Checking Explanations (2020.acl-main)
Copied to clipboard
| Challenge: | Existing work on automated fact checking is concerned with predicting the veracity of claims based on metadata, social network spread, language used in claims, and, more recently, evidence supporting or denying claims. |
| Approach: | They propose to combine the generation of justifications for verdicts on claims with the multi-task model to optimize both objectives at the same time rather than training them separately. |
| Outcome: | The proposed model improves the informativeness, coverage and overall quality of the generated explanations, rather than training them separately. |
Evidence-based Fact-Checking of Health-related Claims (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evidence-based factchecking datasets contain synthetic claims and lack real-world verification. |
| Approach: | They propose a dataset for evidence-based fact-checking of health-related claims that evaluates their truthfulness against scientific articles. |
| Outcome: | The proposed dataset evaluates real-world claims against scientific articles. |