PoliBERTweet: A Pre-trained Language Model for Analyzing Political Content on Twitter (2022.lrec-1)
Copied to clipboard
| Challenge: | Pre-trained domain-specific models are useful for understanding domain-level contexts. |
| Approach: | They propose to use a pre-trained language model to better capture domain-specific contexts. |
| Outcome: | The proposed model outperforms general-purpose models on election-related tasks. |
Similar Papers
BERTweet: A pre-trained language model for English Tweets (2020.emnlp-demos)
Copied to clipboard
| Challenge: | Experiments show that BERTweet outperforms strong baselines RoBERTa-base and XLM-R-base on three Tweet NLP tasks: Part-of-speech tagging, Named-entity recognition and text classification. |
| Approach: | They propose to train a pre-trained language model for English Tweets using the RoBERTa pre training procedure and use it to train the model. |
| Outcome: | Experiments show that the model outperforms baseline models on three Tweet NLP tasks: Part-of-speech tagging, Named-entity recognition and text classification. |
TweetNLP: Cutting-Edge Natural Language Processing for Social Media (2022.emnlp-demos)
Copied to clipboard
Jose Camacho-collados, Kiamehr Rezaee, Talayeh Riahi, Asahi Ushio, Daniel Loureiro, Dimosthenis Antypas, Joanne Boisson, Luis Espinosa Anke, Fangyu Liu, Eugenio Martínez Cámara
| Challenge: | TweetNLP is an integrated platform for natural language processing in social media. |
| Approach: | They propose a Python-based platform for natural language processing in social media that supports a variety of NLP tasks. |
| Outcome: | The proposed platform supports generic focus areas such as sentiment analysis and named entity recognition, as well as social media-specific tasks such as emoji prediction and offensive language identification. |
ConfliBERT: A Pre-trained Language Model for Political Conflict and Violence (2022.naacl-main)
Copied to clipboard
Yibo Hu, MohammadSaleh Hosseini, Erick Skorupa Parolin, Javier Osorio, Latifur Khan, Patrick Brandt, Vito D’Orazio
| Challenge: | Traditionally, researchers used manual coding to track conflict processes worldwide, but the high costs and slow pace of domain experts make it difficult and costly to monitor complex and rapidly changing conflicts. |
| Approach: | They propose a domain-specific pre-trained language model for conflict and political violence that can be used to train a language model from scratch and continue training. |
| Outcome: | The proposed model outperforms BERT in conflict research. |
From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models (2023.acl-long)
Copied to clipboard
| Challenge: | Hundreds of studies have highlighted ethical issues in NLP models . |
| Approach: | They propose to measure media biases in LMs trained on diverse data sources . they focus on hate speech and misinformation detection . |
| Outcome: | The proposed methods quantify the fairness of downstream NLP models trained on politically biased LMs. |
IndoBERTweet: A Pretrained Language Model for Indonesian Twitter with Effective Domain-Specific Vocabulary Initialization (2021.emnlp-main)
Copied to clipboard
| Challenge: | In IndoBERTweet, a pretraining model for Indonesian Twitter is extended with domain-specific vocabulary. |
| Approach: | They propose a pretraining model that extends a monolingual Indonesian BERT model with domain-specific vocabulary. |
| Outcome: | The proposed model can be initialized with the average BERT subword embedding five times faster than existing methods for vocabulary adaptation. |
TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Modern NLP systems are typically ill-equipped when applied to noisy user-generated text. |
| Approach: | They propose a new evaluation framework consisting of seven Twitter-specific classification tasks. |
| Outcome: | The proposed framework is based on seven heterogeneous Twitter-specific classification tasks. |
A Million Tweets Are Worth a Few Points: Tuning Transformers for Customer Service Tasks (2021.naacl-main)
Copied to clipboard
| Challenge: | In domain-specific customer service applications, many companies struggle to deploy advanced NLP models due to the limited availability of and noise in their datasets. |
| Approach: | They analyze customer service conversations on a multilingual social media corpus and compare different approaches to pretraining and finetuning on different end tasks. |
| Outcome: | The proposed model improves performance on multilingual social media data, especially in non-English settings. |
Bernice: A Multilingual Pre-trained Encoder for Twitter (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing language models for Twitter are monolingual, adapted from other domains, or trained on limited amount of in-domain data. |
| Approach: | They propose a multilingual RoBERTa language model that is trained from scratch on 2.5 billion tweets with a custom tweet-focused tokenizer. |
| Outcome: | The proposed model outperforms or matches models trained on monolingual and multilingual tweets on a variety of benchmarks and is more efficient compute- and data-wise to train completely on in-domain data with a specialized domain-specific tokenizer. |
Developing A Multilabel Corpus for the Quality Assessment of Online Political Talk (2022.lrec-1)
Copied to clipboard
| Challenge: | a corpus of political tweets labeled for its deliberative characteristics is presented . the dataset offers a first step in building dictionaries to aid in the measurement of the Discourse Quality Index . |
| Approach: | They present a Twitter Deliberative Politics dataset that measures the quality of political tweets . they propose to use machine learning to analyze tweets and to use it to build dictionaries . |
| Outcome: | The proposed dataset is useful to linguists, political scientists, and social scientists . it offers a first step in building dictionaries for the quality assessment of political talk in english . |
Annotating the Tweebank Corpus on Named Entity Recognition and Building NLP Models for Social Media Analysis (2022.lrec-1)
Copied to clipboard
| Challenge: | Social media data such as Twitter messages pose a particular challenge to NLP systems because of their short, noisy nature. |
| Approach: | They create a Twitter-based NER corpus and train Tweet NLP models on it . they annotate named entities in TB2 using Amazon Mechanical Turk . |
| Outcome: | The proposed model outperforms existing models on Twitter and other social media platforms. |