QA-NatVer: Question Answering for Natural Logic-based Fact Verification (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has focused on natural logic, which operates directly on natural language by capturing the semantic relation of spans between an aligned claim and its evidence via set-theoretic operators. |
| Approach: | They propose to use question answering to predict natural logic operators using generalization capabilities of instruction-tuned language models. |
| Outcome: | The proposed approach outperforms the best baseline on a Danish verification dataset by 4.3 accuracy points. |
Similar Papers
ProoFVer: Natural Logic Theorem Proving for Fact Verification (2022.tacl-1)
Copied to clipboard
| Challenge: | Recent fact verification systems rely on neural network classifiers for veracity prediction, which lack explainability. |
| Approach: | They propose a model that generates natural logic-based inferences as proofs using lexical mutations between spans in the claim and the evidence retrieved. |
| Outcome: | The proposed model has highest label accuracy and second best score in the FEVER leaderboard. |
Can NLI Models Verify QA Systems’ Predictions? (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent question answering systems perform well on benchmark datasets, but are not always well-calibrated to spot spurious answers under distribution shifts. |
| Approach: | They propose to use natural language inference to verify whether answers are correct . they leverage large pre-trained models and recent prior datasets to construct powerful question conversion and decontextualization modules. |
| Outcome: | The proposed approach improves the confidence estimation of a QA model across different domains, evaluated in a selective QA setting. |
Zero-Shot Fact Verification via Natural Logic and Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advancements in fact verification systems with natural logic have enhanced their explainability by aligning claims with evidence through set-theoretic operators, providing faithful justifications. |
| Approach: | They propose a method that utilizes the generalization capabilities of instruction-tuned large language models to provide faithful justifications. |
| Outcome: | The proposed method outperforms other systems that were not specifically trained on natural logic data, and achieves an average accuracy improvement of 8.96 points over the baseline. |
Complex Reasoning in Natural Language (2023.acl-tutorials)
Copied to clipboard
| Challenge: | Recent research shows that pretrained language models are often brittle for complex reasoning tasks. |
| Approach: | They propose to use pre-trained language models to teach machines to reason over texts . they will review recent promising approaches to tackling complex reasoning tasks . |
| Outcome: | This tutorial reviews promising approaches to complex reasoning tasks . it reviews the methods that can be used to augment models with robustness . |
Automated Fact-Checking in Dialogue: Are Specialized Models Needed? (2023.emnlp-main)
Copied to clipboard
| Challenge: | Prior work has shown that typical fact-checking models struggle with claims made in conversation. |
| Approach: | They propose to fine-tune models for dialogue on conversational data to improve performance on typical fact-checking. |
| Outcome: | The proposed models perform better on stand-alone claims than state-of-the-art models for dialogue while maintaining their performance on standalone claim. |
Neural Natural Logic Inference for Interpretable Question Answering (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing question answering models are based on textual entailment tasks . prior work has focused on QA on premise-based questions . |
| Approach: | They propose a neural-symbolic QA approach that integrates natural logic reasoning within deep learning architectures towards developing effective question answering models. |
| Outcome: | The proposed model outperforms previous work on multiple-choice science questions . it integrates natural logic reasoning within deep learning architectures to build proof paths . |
QaDialMoE: Question-answering Dialogue based Fact Verification with Mixture of Experts (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing research on fact verification focuses on news, tables and Wikipedia passages. |
| Approach: | They propose a question-answering dialogue based fact verification with mixture of experts that exploits questions and evidence effectively in the verification process. |
| Outcome: | The proposed approach outperforms previous approaches on three benchmark datasets and achieves state-of-the-art results. |
A Survey on Automated Fact-Checking (2022.tacl-1)
Copied to clipboard
| Challenge: | Fact-checking is an essential task in journalism due to the speed with which information and misinformation can spread in the media ecosystem. |
| Approach: | They propose to use natural language processing to automate fact-checking by identifying common concepts and defining definitions. |
| Outcome: | The proposed method can predict the veracity of claims using natural language processing, machine learning, and databases. |
Using contradictions improves question answering systems (2023.acl-short)
Copied to clipboard
| Challenge: | Existing systems that use contradiction to determine if a question is supported by background contexts do better than those that use entailment. |
| Approach: | They propose a method that incorporates contradiction in natural language inference (NLI) they propose to reformulate answers from QA systems as hypotheses and then select the best one based on the results. |
| Outcome: | The proposed method improves on multiple choice and extractive QA in two settings. |
Logic-Guided Data Augmentation and Regularization for Consistent Question Answering (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies have focused on improving the consistency of comparison questions . current models show inconsistent comparison predictions due to small datasets . |
| Approach: | They propose a method that integrates logic rules and neural models to improve the accuracy of comparison questions by enhancing labeled training data. |
| Outcome: | The proposed method improves state-of-the-art models on WIQA and QuaRel and reduces consistency violations by 58% on HotpotQA. |