Proceedings of the Second Workshop on Natural Language Processing for Internet Freedom: Censorship, Disinformation, and Propaganda
Assessing Post Deletion in Sina Weibo: Multi-modal Classification of Hot Topics (D19-50)
Copied to clipboard
Meisam Navaki Arefi, Rajkumar Pandi, Michael Carl Tschantz, Jedidiah R. Crandall, King-wa Fu, Dahlia Qiu Shi, Miao Sha
| Challenge: | Weibo monitors and deletes posts to conform to government requirements . a recent study found that sentiment is the only indicator of censorship that is consistent across topics . |
| Approach: | They analyze a dataset of censored and uncensore censors on Weibo . they use deep learning, CNN localization, and NLP techniques to analyze the data . |
| Outcome: | The proposed analysis of censored and uncensoreded posts in Weibo shows that sentiment is the only indicator of a topic's censorship . censors can remove posts that are considered sensitive in three hours on average . |
Detecting context abusiveness using hierarchical deep learning (D19-50)
Copied to clipboard
| Challenge: | Abusive text is a serious problem in social media and causes many issues among users . a model that detects text abusiveness in context without explicit abusive words is challenging . |
| Approach: | They propose to use an abusive lexicon to determine the existence of an abusive word in text . they combine local and global features to evaluate the model using benchmark data . |
| Outcome: | The proposed model outperforms all previous models for detecting abusiveness in text without abusive words. |
How Many Users Are Enough? Exploring Semi-Supervision and Stylometric Features to Uncover a Russian Troll Farm (D19-50)
Copied to clipboard
| Challenge: | Social media has been used by troll farms to promote political agendas . trolled farms employ people to provoke conflict via the use of inflammatory or provocative comments. |
| Approach: | They analyze the use of self-supervision with less than 100 troll accounts as training data to determine whether a trolled account is labeled as a Russian trol farm. |
| Outcome: | The proposed methods improve classification performance by nearly 4% F1 and use self-supervision with less than 100 troll accounts as training data. |
Identifying Nuances in Fake News vs. Satire: Using Semantic and Linguistic Cues (D19-50)
Copied to clipboard
| Challenge: | a blurry line between fake news and protected-speech satire has been a struggle for social media platforms . purveyors of fake news have begun to masquerade as satirical sites to avoid being demoted . |
| Approach: | They propose to automatically classify fake news versus satire based on language differences . they hypothesize that nuances could be identified using semantic and linguistic cues . |
| Outcome: | The proposed method can identify nuances between fake news and satire based on language differences . the proposed method is compared to the language-based baseline and is highly scalable . |
Calls to Action on Social Media: Detection, Social Impact, and Censorship Potential (D19-50)
Copied to clipboard
| Challenge: | Calls to action are effective means of mobilization in social networks, but their potential for censorship and predicting offline protest events has not yet been evaluated. |
| Approach: | They examine the possibility of their automatic detection on historical data from the 2011-2013 protests in Bolotnaya, Russia. |
| Outcome: | The political calls to action can be annotated and detected with relatively high accuracy and have a moderate positive correlation with actual rally attendance. |
Mapping (Dis-)Information Flow about the MH17 Plane Crash (D19-50)
Copied to clipboard
| Challenge: | Digital media enables fast sharing of information, but also disinformation . studies on the spread of disinformation on social media focused on small, manually annotated datasets or used proxys for data annotation. |
| Approach: | They propose to use text classifiers to label Twitter content related to the MH17 crash to improve annotation accuracy. |
| Outcome: | The proposed classifier improves over a hashtag-based baseline, but still remains a challenge in labelling pro-Russian and pro-Ukrainian content with high precision. |
Generating Sentential Arguments from Diverse Perspectives on Controversial Topic (D19-50)
Copied to clipboard
| Challenge: | ArgDiver model generates high-quality sentential arguments from multiple perspectives . retrieval-based systems do not have sufficient flexibility for input with missing keywords or topics unseen . |
| Approach: | They propose a neural method to generate sentential arguments from multiple perspectives . their model generates high-quality sentential argument, but shows higher diversity . |
| Outcome: | The proposed model generates high-quality sentential arguments from multiple perspectives . it shows that it can provide diverse perspectives on a controversial topic . |
Rumor Detection on Social Media: Datasets, Methods and Opportunities (D19-50)
Copied to clipboard
| Challenge: | Social media platforms are used for information gathering, but they also lead to the spreading of rumors and fake news. |
| Approach: | This paper presents a comprehensive list of datasets used for rumor detection . it also reviews the important studies based on what types of information they exploit . |
| Outcome: | This paper presents an overview of the recent studies in the rumor detection field . it provides a comprehensive list of datasets used for rumour detection . |
Unraveling the Search Space of Abusive Language in Wikipedia with Dynamic Lexicon Acquisition (D19-50)
Copied to clipboard
| Challenge: | Existing methods to detect abusive language only train one classifier for the whole variety of offending . a new method can support a moderator with explicit unraveled explanations for why something was flagged as abusive . |
| Approach: | a new method is proposed to distinguish explicitly abusive cases from the more "shadowed" ones . the researchers extend a lexicon of abusive terms to include new obfuscations of abusive words . |
| Outcome: | a new method can distinguish explicitly abusive cases from the more "shadowed" ones . the method can support a moderator with explicit unraveled explanations for why something was flagged as abusive . |
CAUnLP at NLP4IF 2019 Shared Task: Context-Dependent BERT for Sentence-Level Propaganda Detection (D19-50)
Copied to clipboard
| Challenge: | Sentence-level and fragment-level propaganda detection tasks are more challenging compared to document-level detection. |
| Approach: | They propose to use context-dependent input pairs to fine-tune the pretrained propaganda detection BERT to better utilize document information. |
| Outcome: | The proposed system can detect propaganda on document-level, sentence-level and fragment-level. |
Fine-Grained Propaganda Detection with Fine-Tuned BERT (D19-50)
Copied to clipboard
| Challenge: | The goal of the Fragment Level Classification task is to detect and classify textual segments that correspond to one of the 18 given propaganda techniques in a news articles dataset. |
| Approach: | They propose a model that performs word-level classification using a pre-trained language model to detect and classify propaganda fragments in a news article dataset. |
| Outcome: | The proposed model performs word-level classification using a popular pre-trained language model. |
Neural Architectures for Fine-Grained Propaganda Detection in News (D19-50)
Copied to clipboard
| Challenge: | MIC-CIS is a fine grained propaganda detection system . previous work focused on document level, labeling articles as propaganda . |
| Approach: | They propose to use different neural architectures to jointly perform propaganda detection tasks . they also investigate different ensemble schemes such as majority-voting, relax-vote, etc. |
| Outcome: | The proposed system performs sentences and fragment level propaganda detection tasks. |
Fine-Tuned Neural Models for Propaganda Detection at the Sentence and Fragment levels (D19-50)
Copied to clipboard
| Challenge: | The system was evaluated on a unified development set without distributing the gold labels. |
| Approach: | They propose to use fine-grained propaganda detection to build models that can explain why an article is propagandistic. |
| Outcome: | The proposed model performed on all eighteen propaganda techniques in the corpus of the shared task. |
Divisive Language and Propaganda Detection using Multi-head Attention Transformers with Deep Learning BERT-based Language Models for Binary Classification (D19-50)
Copied to clipboard
| Challenge: | a team of researchers used a pre-trained BERT language model to train propaganda . the model was based on a cloze comprehension test to answer a question about influence operations . |
| Approach: | team used a BERT language model that was pre-trained on Wikipedia and BookCorpus . they used cloze comprehension tests to train the model to answer a propaganda question . |
| Outcome: | The proposed model was trained on Wikipedia and BookCorpus to answer propaganda questions . the team used a neural network that was pre-trained on the Wikipedia and bookCorpus corpus . |
On Sentence Representations for Propaganda Detection: From Handcrafted Features to Word Embeddings (D19-50)
Copied to clipboard
| Challenge: | The rise of fake and hyperpartisan news on social media and online news outlets calls for improved automatic detection of propaganda in texts. |
| Approach: | They propose to use handcrafted features and learn dense semantic representations to detect propaganda in sentence-level and with random undersampling of the majority class (non-propaganda) |
| Outcome: | The proposed system achieves a ranking of 10 among 25 participants, with 59.5 F1-score. |
JUSTDeep at NLP4IF 2019 Task 1: Propaganda Detection using Ensemble Deep Learning Models (D19-50)
Copied to clipboard
| Challenge: | Detecting fake news is not well established yet, but it can be classified under several labels: false, biased, or framed to mislead the readers. |
| Approach: | They propose a deep learning model using BiLSTM, XGBoost, and BERT to detect propaganda using a corpus from a challenge. |
| Outcome: | The proposed model outperforms the baseline model on a dataset from the challenge NLP4IF 2019 . |
Detection of Propaganda Using Logistic Regression (D19-50)
Copied to clipboard
| Challenge: | Various propaganda techniques are used to manipulate peoples perspectives to foster a predetermined agenda. |
| Approach: | They propose a Logistic Regression-based tool that automatically classifies whether a sentence is propagandistic or not. |
| Outcome: | The proposed tool outperforms the baseline on linguistic and semantic features. |
Cost-Sensitive BERT for Generalisable Sentence Classification on Imbalanced Data (D19-50)
Copied to clipboard
| Challenge: | Popular NLP tasks such as sentiment analysis and event extraction from social media are examples of imbalanced classification problems. |
| Approach: | They propose a method to generalise on dissimilar training and test data using a measure of similarity between datasets. |
| Outcome: | The proposed method achieves the second highest score on sentence-level propaganda classification. |
Understanding BERT performance in propaganda analysis (D19-50)
Copied to clipboard
| Challenge: | Despite the challenging nature of the shared task, our pretrained BERT model scored 0.62 F1 on the test set and ranked third among 25 teams who participated in the contest. |
| Approach: | They propose to use a dataset to fine-tune a model for propaganda analysis at sentence level to determine whether a text is 'propaganda' and to examine false-positive cases. |
| Outcome: | The proposed model scored 0.62 F1 on the test set and ranked third among 25 teams who participated in the shared task. |
Pretrained Ensemble Learning for Fine-Grained Propaganda Detection (D19-50)
Copied to clipboard
| Challenge: | Propaganda detection is a reallife challenge that can affect how people understand news . |
| Approach: | They propose to use a manually annotated dataset to tackle the propaganda detection on sentence level classification task of NLP4IF 2019 workshop co-located with EMNLP-IJCNLP 2019 conference. |
| Outcome: | The proposed model is ranked in the first place with 68.8312 F1-score on the development dataset and in the sixth place with 61.3990 F1 score on the testing dataset. |
NSIT@NLP4IF-2019: Propaganda Detection from News Articles using Transfer Learning (D19-50)
Copied to clipboard
| Challenge: | In this paper, we describe our approach and system description for NLP4IF 2019 Workshop: Shared Task on Fine-Grained Propaganda Detection. |
| Approach: | They propose to use document Embeddings and LSTM to detect whether a sentence contains a propagandistic agenda. |
| Outcome: | The proposed approach ranked 21st in the NLP4IF 2019 Workshop: Shared Task on Fine-Grained Propaganda Detection. |
Sentence-Level Propaganda Detection in News Articles with Transfer Learning and BERT-BiLSTM-Capsule Model (D19-50)
Copied to clipboard
| Challenge: | a new task is needed to detect propaganda in news articles . the need for communication has increased in online social media platforms . a proposed solution to the problem of sentence-level propaganda classification is ranked 12th . |
| Approach: | They propose to build a binary classifier able to provide corresponding propaganda labels . their solution ranks 12th among 26 teams in the NLP4IF-2019 Shared Task SLC . |
| Outcome: | The proposed model outperforms baseline approach and the winning system on a similar task. |
Synthetic Propaganda Embeddings To Train A Linear Projection (D19-50)
Copied to clipboard
| Challenge: | Using contextualized token embeddings, we can extract features of propaganda from contextualized embeddnings without fine-tuning the large parameters of the base model. |
| Approach: | They propose a method for detecting fine-grained categories of propaganda in text by generating synthetically generated embeddings from pre-trained language models. |
| Outcome: | The proposed method is used in the first shared task in fine-grained propaganda detection at NLP4IF as Team Stalin. |
Findings of the NLP4IF-2019 Shared Task on Fine-Grained Propaganda Detection (D19-50)
Copied to clipboard
| Challenge: | A shared task on fine-grained propaganda detection was organized at EMNLP-IJCNLP 2019 . 12 systems submitted systems for the FLC task, 25 for the SLC task, and 14 teams submitted a system description paper . |
| Approach: | They present a task on fine-grained propaganda detection as part of the NLP4IF workshop at EMNLP-IJCNLP 2019 . they used a corpus of news articles annotated with an inventory of propagandist techniques at the fragment level to determine the propaganda technique used in each fragment . |
| Outcome: | The shared task on fine-grained propaganda detection was organized at the EMNLP-IJCNLP 2019 . 12 systems submitted for the FLC task, 25 for the SLC task, and 14 submitted a system description paper . |