Automated Fact-Checking in Dialogue: Are Specialized Models Needed? (2023.emnlp-main)
Copied to clipboard
| Challenge: | Prior work has shown that typical fact-checking models struggle with claims made in conversation. |
| Approach: | They propose to fine-tune models for dialogue on conversational data to improve performance on typical fact-checking. |
| Outcome: | The proposed models perform better on stand-alone claims than state-of-the-art models for dialogue while maintaining their performance on standalone claim. |
Similar Papers
DialFact: A Benchmark for Fact-Checking in Dialogue (2022.acl-long)
Copied to clipboard
| Challenge: | Existing fact-checking models trained on non-dialogue data fail to perform well on this task. |
| Approach: | They propose a task of fact-checking in dialogue to improve fact- checking performance . they propose to use an annotated conversational claim and Wikipedia snippets as evidence . |
| Outcome: | The proposed task improves fact-checking performance in dialogue. |
A Survey on Automated Fact-Checking (2022.tacl-1)
Copied to clipboard
| Challenge: | Fact-checking is an essential task in journalism due to the speed with which information and misinformation can spread in the media ecosystem. |
| Approach: | They propose to use natural language processing to automate fact-checking by identifying common concepts and defining definitions. |
| Outcome: | The proposed method can predict the veracity of claims using natural language processing, machine learning, and databases. |
How Robust are Fact Checking Systems on Colloquial Claims? (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing fact checking systems that perform well on colloquial claims significantly degenerate on collotic claims with the same semantics. |
| Approach: | They propose to transfer the styles of claims from FEVER into colloquialism to investigate fact checking systems on colloqual claims. |
| Outcome: | The proposed system significantly degenerates on colloquial claims with the same semantics. |
Automated Fact Checking: Task Formulations, Methods and Future Directions (C18-1)
Copied to clipboard
| Challenge: | Recent research on fact checking has focused on misinformation . however, relevant papers and articles have been published in research communities that are unaware of each other and use inconsistent terminology. |
| Approach: | They propose avenues for future NLP research on automated fact checking . they highlight the use of evidence as an important distinguishing factor . |
| Outcome: | The proposed methods unify the task formulations and methodologies across papers and authors. |
Explainable Automated Fact-Checking: A Survey (2020.coling-main)
Copied to clipboard
| Challenge: | Steady progress has been made in fact-checking and its orthogonal tasks. |
| Approach: | They propose to use fact-checking explanations to explain predictions by comparing existing explanations against desirable properties to find out what makes for good explanations. |
| Outcome: | The proposed explanations are compared against desirable properties and show how they may lead to improvements in the research area. |
QA-NatVer: Question Answering for Natural Logic-based Fact Verification (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has focused on natural logic, which operates directly on natural language by capturing the semantic relation of spans between an aligned claim and its evidence via set-theoretic operators. |
| Approach: | They propose to use question answering to predict natural logic operators using generalization capabilities of instruction-tuned language models. |
| Outcome: | The proposed approach outperforms the best baseline on a Danish verification dataset by 4.3 accuracy points. |
Can NLI Models Verify QA Systems’ Predictions? (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent question answering systems perform well on benchmark datasets, but are not always well-calibrated to spot spurious answers under distribution shifts. |
| Approach: | They propose to use natural language inference to verify whether answers are correct . they leverage large pre-trained models and recent prior datasets to construct powerful question conversion and decontextualization modules. |
| Outcome: | The proposed approach improves the confidence estimation of a QA model across different domains, evaluated in a selective QA setting. |
Exploring Strategies for Generalizable Commonsense Reasoning with Pre-trained Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work proposes lightweight updates to improve commonsense reasoning models . fine-tuning can cause models to overfit to task-specific data and forget knowledge gained during training . |
| Approach: | They propose to use lightweight models to update pre-trained language models to learn commonsense background knowledge. |
| Outcome: | The proposed models learn from commonsense reasoning datasets, but they are overfitted and limited generalized. |
Assisting the Human Fact-Checkers: Detecting All Previously Fact-Checked Claims in a Document (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent years have brought us a proliferation of false claims online, which spread fast . fact-checkers have been using automated fact-finding to verify claims . |
| Approach: | They propose a system that can detect claims that can be fact-checked by a given database . they create a manually annotated document dataset and propose evaluation measures . |
| Outcome: | The proposed system achieves sizable performance gains over strong baselines. |
Automatic Fake News Detection: Are Models Learning to Reason? (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for fake news detection rely on reasoning . existing work has not explored the predictive power of isolated evidence . |
| Approach: | They investigate the relationship and importance of both claim and evidence in fact checking models. |
| Outcome: | The proposed model performs better on political fact checking datasets using both the claim and evidence. |