| Challenge: | Existing objective functions for fact verification fail to capture heterogeneity among verdict classes . cross-entropy loss treats all misclassification types uniformly, which is problematic . |
| Approach: | They propose two task-specific objective functions that capture the heterogeneity among verdict classes . they use a dictionary-based objective function to classify Wikipedia sentences into three verdict classes. |
| Outcome: | The proposed objectives outperform the standard cross-entropy loss objective . the proposed objectives are combined with simple class weighting to overcome imbalance . |
Similar Papers
The FEVER2.0 Shared Task (D19-66)
Copied to clipboard
| Challenge: | Existing deep neural models are becoming more complex and difficult to understand and characterize their behaviour. |
| Approach: | They present the results of the second Fact Extraction and VERification (FEVER2.0) Shared Task. |
| Outcome: | The proposed task was based on the second Fact Extraction and VERification (FEVER2.0) shared task. |
Team GPLSI. Approach for automated fact checking (D19-66)
Copied to clipboard
| Challenge: | Automated fact checking is a task for proving news veracity by reliable sources. |
| Approach: | They propose to use triplets to extract sentences and compare them to Wikipedia articles using semantic similarity. |
| Outcome: | The proposed approach is satisfactory but there is room for improvement. |
Constrained Fact Verification for FEVER (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for fact verification rely on extracted evidence, but there is little work on understanding the reasoning process. |
| Approach: | They propose a method that enforces a closed-world reliance on extracted evidence to verify a claim's factuality. |
| Outcome: | The proposed model outperforms existing models on the FEVER shared task and shows that it is more accurate than previous models. |
Dice Loss for Data-imbalanced NLP Tasks (2020.acl-main)
Copied to clipboard
| Challenge: | Using dice loss, we find that data imbalance is a common issue in many NLP tasks . data imbalance affects the performance of many tasks, such as tagging and machine reading comprehension . |
| Approach: | They propose to use dice loss to replace the standard cross-entropy objective for data-imbalanced NLP tasks. |
| Outcome: | The proposed training objective achieves significant performance boost on a wide range of data imbalanced tasks. |
DeSePtion: Dual Sequence Prediction and Adversarial Examples for Improved Fact-Checking (2020.acl-main)
Copied to clipboard
Christopher Hidey, Tuhin Chakrabarty, Tariq Alhindi, Siddharth Varia, Kriste Krstovski, Mona Diab, Smaranda Muresan
| Challenge: | Fact Extraction and Verification datasets provide a resource for end-to-end fact-checking, requiring retrieval of evidence from Wikipedia to validate a veracity prediction. |
| Approach: | They propose a system that is resilient to attacks by multiple propositions, temporal reasoning, ambiguity and lexical variation and a sequence of evidence sentences and veracity relation predictions. |
| Outcome: | The proposed system is resilient to three realistic “attacks” and obtains state-of-the-art results due to improved evidence retrieval. |
FEVER Breaker’s Run of Team NbAuzDrLqg (D19-66)
Copied to clipboard
| Challenge: | In the second workshop on Fact Extraction and VERification, the goal is to develop a fact-check system which can resolve "fake news" and misinformation problems. |
| Approach: | They propose to use a model to retrieve evidence when appropriate query terms could not be easily generated from the claim. |
| Outcome: | The proposed models were able to get both the evidence and label correct in 20% of the data. |
Team DOMLIN: Exploiting Evidence Enhancement for the FEVER Shared Task (D19-66)
Copied to clipboard
| Challenge: | Existing methods of fact checking are based on the assignment of a truth value to a given (factual) statement, and therefore it is desirable to have access to the evidence used to reach an assignment. |
| Approach: | They propose a two-staged sentence selection strategy to account for examples in the dataset where evidence is not only conditioned on the claim, but also on previously retrieved evidence. |
| Outcome: | The proposed system beats the top performing systems of the first FEVER challenge which act as a baseline, beating 64.21% of the top-performing systems. |
FEVER: a Large-scale Dataset for Fact Extraction and VERification (N18-1)
Copied to clipboard
| Challenge: | 185,445 claims generated by altering sentences from Wikipedia are verified without knowledge of the sentence they were derived from. |
| Approach: | They propose a publicly available dataset for verification against textual sources, FEVER: Fact Extraction and VERification. |
| Outcome: | The proposed dataset achieves 31.87% accuracy on labeling a claim accompanied by the correct evidence, compared to 50.91% if we ignore the evidence. |
Unified Interpretation of Softmax Cross-Entropy and Negative Sampling: With Case Study for Knowledge Graph Embedding (2021.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge graph embedding methods do not provide a fairly accurate comparison of the two loss functions. |
| Approach: | They propose to use the Bregman divergence to provide a unified interpretation of the softmax cross-entropy and negative sampling loss functions. |
| Outcome: | The proposed model can be used to predict missing relational links between entities using a scoring method. |
Exploring Listwise Evidence Reasoning with T5 for Fact Verification (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for fact verification use pretrained sequence-to-sequence transformers for sentence selection and label prediction. |
| Approach: | They propose a framework for fact verification that leverages pretrained sequence-to-sequence transformer models for sentence selection and label prediction. |
| Outcome: | The proposed framework scores higher than the second place approach on the blind test set . the proposed framework can be useful for a broader range of NLP tasks, the authors say . |