Dense Feature Memory Augmented Transformers for COVID-19 Vaccination Search Classification (2022.emnlp-industry)
Copied to clipboard
Jai Gupta, Yi Tay, Chaitanya Kamath, Vinh Tran, Donald Metzler, Shailesh Bavadekar, Mimi Sun, Evgeniy Gabrilovich
| Challenge: | a new method for classification of COVID-19 vaccination related search queries is proposed . the proposed method uses pretrained Transformers and dense features to generate search insights . |
| Approach: | They propose a machine learning model that detects COVID-19 vaccination related search queries . they use pretrained Transformers to consider dense features as memory tokens that the model can attend to . |
| Outcome: | The proposed model improves the Vaccine Search Insights task by +15% . the proposed model uses pretrained Transformers and traditional dense features . |
Similar Papers
VaccineLies: A Natural Language Resource for Learning to Recognize Misinformation about the COVID-19 and HPV Vaccines (2022.lrec-1)
Copied to clipboard
| Challenge: | VaccineLies can detect misinformation about vaccines on Twitter without using language resources. |
| Approach: | They present a dataset of tweets propagating misinformation about two vaccines . authors propose novel methods to detect misinformation on Twitter and identify stance towards it . |
| Outcome: | VaccineLies can detect misinformation on Twitter and identify the stance towards it. |
Forecasting COVID-19 Caseloads Using Unsupervised Embedding Clusters of Social Media Posts (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing studies have shown that social media can help predict rises in infectious disease caseloads. |
| Approach: | They propose to use transformer-based language models to integrate infectious disease modelling into reddit embedding features in reddits in specific US states. |
| Outcome: | The proposed model outperforms other features at predicting upward trend signals in areas where epidemiological data is unreliable. |
CoNTACT: A Dutch COVID-19 Adapted BERT for Vaccine Hesitancy and Argumentation Detection (2022.coling-1)
Copied to clipboard
| Challenge: | CoNTACT is a Dutch language model adapted to the domain of COVID-19 tweets . a turbulent vaccine debate has emerged between advocates and opponents of vaccines - a polarization that will continue to influence future views on vaccines. |
| Approach: | They propose a Dutch language model adapted to the domain of COVID-19 tweets . they use 2.8M Dutch COVId-19 related tweets posted in 2021 to test the model . |
| Outcome: | The proposed model shows statistically significant gains over RobBERT on two tasks. |
Boosting Transformers and Language Models for Clinical Prediction in Immunotherapy (2023.acl-industry)
Copied to clipboard
| Challenge: | Current machine learning approaches to predict clinical outcomes are limited to tabular data and are not applicable to clinical prediction. |
| Approach: | They investigate the potential of transformers to improve clinical prediction compared to conventional machine learning approaches and address the challenge of few-shot learning in predicting rare disease areas. |
| Outcome: | The proposed model improves the accuracy of baseline models and language models under few-shot regimes and shows that it is more accurate than previous models. |
Extracting a Knowledge Base of COVID-19 Events from Social Media (2022.coling-1)
Copied to clipboard
| Challenge: | a flood of COVID-19 related information has appeared on social media since December 2019 . this includes reports on public figures who have tested positive/negative for the virus . |
| Approach: | They construct a corpus of 10,000 tweets with annotated public reports of five COVID-19 events, using slot-filling questions to fill in slots. |
| Outcome: | The proposed method can be quickly applied to develop knowledge bases for new domains in response to emerging crises, including natural disasters or future disease outbreaks. |
Examining Temporalities on Stance Detection towards COVID-19 Vaccination (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies have highlighted the importance of vaccination as an effective strategy to control the transmission of the COVID-19 virus. |
| Approach: | They evaluate a range of transformer-based models using chronological and random splits of social media data to examine the impact of temporal concept drift on stance detection towards COVID-19 vaccination. |
| Outcome: | The proposed models show that the models performed better with chronological and random splits than with random split models. |
A Domain Knowledge Enhanced Pre-Trained Language Model for Vertical Search: Case Study on Medicinal Products (2022.coling-1)
Copied to clipboard
| Challenge: | Existing pre-trained language models lack medicinal product knowledge for product vertical search. |
| Approach: | They propose a biomedical knowledge enhanced pre-trained language model for medicinal product vertical search using ELECTRA’s replaced token detection (RTD) pre-training. |
| Outcome: | The proposed model improves query-title relevance, query intent classification, and named entity recognition in query. |
Domain- and Task-Adaptation for VaccinChatNL, a Dutch COVID-19 FAQ Answering Corpus and Classification Model (2022.coling-1)
Copied to clipboard
| Challenge: | VaccinChatNL is the first FAQ chatbot with over 12k user queries . it can be used to find the representative question that matches a user's request . |
| Approach: | They build a FAQ chatbot from 50 question-answer pairs and annotate user questions with appropriate or new answer classes. |
| Outcome: | The VaccinChatNL is the first publicly available Dutch FAQ answering corpus with large groups of human-paraphrased questions. |
Navigating the Kaleidoscope of COVID-19 Misinformation Using Deep Learning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Various deep learning models have been successfully employed for this type of NLP task of text classification. |
| Approach: | They propose a mixed-domain transfer learning approach that only captures local context and exhibits poor generalization. |
| Outcome: | The proposed model captures local and global contexts, but lacks generalization . a combination of shallow network-based domain-specific models and convolutional neural networks can extract local and globally context directly from the target data in a hierarchical fashion, enabling it to offer a more generalizable solution. |
Augmenting Zero-Shot Dense Retrievers with Plug-in Mixture-of-Memories (2023.emnlp-main)
Copied to clipboard
| Challenge: | Using mixture-of-memory augmenting to augment language models improves model generalization but with diminishing return. |
| Approach: | They develop a mechanism that augments language models with mixture-of-memory Augmentation (MoMA) they augment strong T5-based retrievers with the option to "plug in" unseen memory at inference time. |
| Outcome: | The proposed model outperforms methods with larger model sizes on the BEIR benchmark and achieves comparable or even better performance than methods relying on target-specific pretraining. |