Proceedings of the Second Workshop on Fact Extraction and VERification (FEVER)
The FEVER2.0 Shared Task (D19-66)
Copied to clipboard
| Challenge: | Existing deep neural models are becoming more complex and difficult to understand and characterize their behaviour. |
| Approach: | They present the results of the second Fact Extraction and VERification (FEVER2.0) Shared Task. |
| Outcome: | The proposed task was based on the second Fact Extraction and VERification (FEVER2.0) shared task. |
Fact Checking or Psycholinguistics: How to Distinguish Fake and True Claims? (D19-66)
Copied to clipboard
| Challenge: | Using psycholinguistic features to distinguish lies from true statements is a difficult task and a problem to be solved. |
| Approach: | They compare psycholinguistic text features with fact checking approaches to distinguish lies from true statements using data from a large ongoing study. |
| Outcome: | The proposed methods outperform both fact checking and human baselines but the accuracy is not high. |
Neural Multi-Task Learning for Stance Prediction (D19-66)
Copied to clipboard
| Challenge: | Existing models for fact checking are limited in size due to limited data available . stance detection is a key component of fact checking for journalists and news agencies . |
| Approach: | They propose to use textual information from existing datasets to improve stance prediction. |
| Outcome: | The proposed model outperforms state-of-the-art systems on a public benchmark dataset by 6.0 and 14.4 points in weighting. |
GEM: Generative Enhanced Model for adversarial attacks (D19-66)
Copied to clipboard
| Challenge: | Our model generated malicious claims that mixed facts from various articles, so it became difficult to classify their truthfulness. |
| Approach: | They used a model that inherited the knowledge of pretrained GPT-2 to generate controlled sentences with some additional control. |
| Outcome: | The proposed model generated malicious claims that mixed facts from Wikipedia articles, making it difficult to classify their truthfulness. |
Aligning Multilingual Word Embeddings for Cross-Modal Retrieval Task (D19-66)
Copied to clipboard
| Challenge: | Existing methods to learn multimodal multilingual embeddings for text and image retrieval tasks are limited to English. |
| Approach: | They propose a new approach to learn multimodal multilingual embeddings for matching images and captions in two languages by combing two existing objective functions and adapting alignment between existing languages. |
| Outcome: | The proposed model achieves state-of-the-art in retrieval and caption-caption tasks while adapting existing language alignments. |
Unsupervised Natural Question Answering with a Small Model (D19-66)
Copied to clipboard
| Challenge: | a recent demonstration of the power of huge language models such as GPT-2 to memorise the answers to factoid questions raises questions about the extent to which knowledge is embedded directly within these large models. |
| Approach: | They propose to use unsupervised learning techniques to add knowledge explicitly without extensive training. |
| Outcome: | The proposed architecture allows for explicit addition of knowledge without extensive training. |
Scalable Knowledge Graph Construction from Text Collections (D19-66)
Copied to clipboard
| Challenge: | Existing open-source solutions for analyzing unstructured text are lacking in the field of knowledge graph construction. |
| Approach: | They propose a scalable open-source platform that "distills" a text collection into a knowledge graph . they scale out the Stanford CoreNLP toolkit via Apache Spark integration . |
| Outcome: | The proposed platform scales out the Stanford CoreNLP toolkit via Apache Spark integration . it extracts mentions and relations from documents and then ingests them into a knowledge graph . |
Relation Extraction among Multiple Entities Using a Dual Pointer Network with a Multi-Head Attention Mechanism (D19-66)
Copied to clipboard
| Challenge: | Existing studies on relation extrac-tion focus on finding only one relation between two entities in a single sentence. |
| Approach: | They propose a relation extraction model based on a dual pointer network with a multi-head attention mechanism that finds n-to-1 subject-object relations by using a forward decoder and a backward decode-r. |
| Outcome: | The proposed model achieves the state-of-the-art performance on the ACE-05 and NYT datasets. |
Unsupervised Question Answering for Fact-Checking (D19-66)
Copied to clipboard
| Challenge: | Recent Deep Learning (DL) models have achieved human-level accuracy on natural language tasks such as question-answering, natural language inference, and textual entailment. |
| Approach: | They propose an unsupervised question-answering based approach for a similar task, fact-checking. |
| Outcome: | The proposed approach achieves label accuracy of 80.2% on the development set and 80.25% on the test set. |
Improving Evidence Detection by Leveraging Warrants (D19-66)
Copied to clipboard
| Challenge: | Existing methods for extracting warrants from a corpus of arguments are lacking in argument detection. |
| Approach: | They propose to extract multiple warrants from an existing corpus of arguments and then aggregate them . they show that the method needs to be improved, but that it can still improve evidence detection. |
| Outcome: | The proposed method can improve the performance of evidence detection by analyzing arguments and aggregating them. |
Hybrid Models for Aspects Extraction without Labelled Dataset (D19-66)
Copied to clipboard
| Challenge: | Existing methods to extract aspects from opinions focus on explicit aspects, but sentences do not state them explicitly. |
| Approach: | They propose to use a dictionary-based approach to identify and extract aspects from opinions . they propose to combine topic modelling and dictionary--based method . |
| Outcome: | The proposed models outperform baseline topic model and dictionary-based approach in 58.70% of the evaluations. |
Extract and Aggregate: A Novel Domain-Independent Approach to Factual Data Verification (D19-66)
Copied to clipboard
| Challenge: | Existing methods to verify information are used to verify factual data . a domain-independent fact checking system can solve the problem entirely or at the individual stages. |
| Approach: | They propose a domain-independent fact checking system that can solve the verification problem entirely or at the individual stages. |
| Outcome: | The proposed model can achieve a score on par with state-of-the-art models based on specific datasets . it can be used to verify the truth or falsity of the fact, the authors say . |
Interactive Evidence Detection: train state-of-the-art model out-of-domain or simple model interactively? (D19-66)
Copied to clipboard
| Challenge: | Evidence detection is a method that can be used to find evidence in a new topic without training data. |
| Approach: | They propose to use large amounts of out-of-domain data to train an evidence detection method . they simulate users who read source documents and label sentences they can use as evidence . |
| Outcome: | The proposed method outperforms state-of-the-art models on large out-of domain data. |
Veritas Annotator: Discovering the Origin of a Rumour (D19-66)
Copied to clipboard
| Challenge: | a growing number of fake news sites are used for spreading fake news . a lack of a reliable data set is limiting the use of machine learning in fact-checking . |
| Approach: | They propose a web application that can detect the origin of a rumour by identifying its source . |
| Outcome: | The proposed application can detect fake news claims with better accuracy than humans . |
FEVER Breaker’s Run of Team NbAuzDrLqg (D19-66)
Copied to clipboard
| Challenge: | In the second workshop on Fact Extraction and VERification, the goal is to develop a fact-check system which can resolve "fake news" and misinformation problems. |
| Approach: | They propose to use a model to retrieve evidence when appropriate query terms could not be easily generated from the claim. |
| Outcome: | The proposed models were able to get both the evidence and label correct in 20% of the data. |
Team DOMLIN: Exploiting Evidence Enhancement for the FEVER Shared Task (D19-66)
Copied to clipboard
| Challenge: | Existing methods of fact checking are based on the assignment of a truth value to a given (factual) statement, and therefore it is desirable to have access to the evidence used to reach an assignment. |
| Approach: | They propose a two-staged sentence selection strategy to account for examples in the dataset where evidence is not only conditioned on the claim, but also on previously retrieved evidence. |
| Outcome: | The proposed system beats the top performing systems of the first FEVER challenge which act as a baseline, beating 64.21% of the top-performing systems. |
Team GPLSI. Approach for automated fact checking (D19-66)
Copied to clipboard
| Challenge: | Automated fact checking is a task for proving news veracity by reliable sources. |
| Approach: | They propose to use triplets to extract sentences and compare them to Wikipedia articles using semantic similarity. |
| Outcome: | The proposed approach is satisfactory but there is room for improvement. |