Papers with BERTweet
BERTweet: A pre-trained language model for English Tweets (2020.emnlp-demos)
Copied to clipboard
| Challenge: | Experiments show that BERTweet outperforms strong baselines RoBERTa-base and XLM-R-base on three Tweet NLP tasks: Part-of-speech tagging, Named-entity recognition and text classification. |
| Approach: | They propose to train a pre-trained language model for English Tweets using the RoBERTa pre training procedure and use it to train the model. |
| Outcome: | Experiments show that the model outperforms baseline models on three Tweet NLP tasks: Part-of-speech tagging, Named-entity recognition and text classification. |
What Clued the AI Doctor In? On the Influence of Data Source and Quality for Transformer-Based Medical Self-Disclosure Detection (2023.eacl-main)
Copied to clipboard
| Challenge: | Recognizing medical self-disclosure is important in many healthcare contexts, but it has been under-explored by the NLP community. |
| Approach: | They analyze a social media-based task to expand existing medical self-disclosure corpus and compare Transformer-based models to determine their merits. |
| Outcome: | The proposed dataset outperforms the state-of-the-art dataset for this task by 16.73%. |
PoliBERTweet: A Pre-trained Language Model for Analyzing Political Content on Twitter (2022.lrec-1)
Copied to clipboard
| Challenge: | Pre-trained domain-specific models are useful for understanding domain-level contexts. |
| Approach: | They propose to use a pre-trained language model to better capture domain-specific contexts. |
| Outcome: | The proposed model outperforms general-purpose models on election-related tasks. |