Sentiment-Aware Word and Sentence Level Pre-training for Sentiment Analysis (2022.emnlp-main)
Copied to clipboard
Shuai Fan, Chen Lin, Haonan Li, Zhenghao Lin, Jinsong Su, Hang Zhang, Yeyun Gong, JIan Guo, Nan Duan
| Challenge: | Existing pre-trained language representation models (PLMs) capture sentiment information from word-level while under-considering sentence-level information. |
| Approach: | They propose a Sentiment-aware pre-trained language model with combined Word-level and Sentence-level Pre-training tasks that enhance the PLM’s knowledge about sentiment words. |
| Outcome: | The proposed model achieves state-of-the-art on various sentence-level and aspect-level sentiment classification benchmarks. |
Similar Papers
SentiLARE: Sentiment-Aware Language Representation Learning with Linguistic Knowledge (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-trained models neglect to consider linguistic knowledge of texts . existing models neglect linguistic information, which is important for sentiment analysis . |
| Approach: | They propose a model that introduces word-level linguistic knowledge into pre-trained models to enhance sentiment analysis by querying SentiWordNet to acquire sentiment polarity. |
| Outcome: | The proposed model obtains state-of-the-art performance on a variety of sentiment analysis tasks. |
SentiX: A Sentiment-Aware Pre-Trained Model for Cross-Domain Sentiment Analysis (2020.coling-main)
Copied to clipboard
| Challenge: | Pre-trained language models have been widely applied to cross-domain NLP tasks like sentiment analysis, but fine-tuning them on the source domain tends to overfit, leading to inferior results on the target domain. |
| Approach: | They propose to pre-train a sentiment-aware language model (SentiX) via domain-invariant sentiment knowledge from large-scale review datasets and utilize it for cross-domain sentiment analysis tasks without fine-tuning. |
| Outcome: | The proposed model achieves state-of-the-art in all the cross-domain sentiment analysis tasks and can be trained with only 1% samples and better than BERT with 90% samples. |
Attentive Gated Lexicon Reader with Contrastive Contextual Co-Attention for Sentiment Classification (D18-1)
Copied to clipboard
| Challenge: | Existing sentiment lexicons do not handle word sense and the concept of semantic compositionality is non-existent in simple lexiconic approaches. |
| Approach: | They propose a lexicon-driven contextual attention mechanism and a contrastive co-attention mechanism that models contrasting polarities between all positive and negative words in a sentence. |
| Outcome: | The proposed model outperforms many other neural baselines on sentiment classification tasks on multiple benchmark datasets. |
Learning Implicit Sentiment in Aspect-based Sentiment Analysis with Supervised Contrastive Pre-Training (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have focused on identifying the sentiment polarity of aspects in product reviews. |
| Approach: | They propose to use supervised Contrastive Pre-Training to learn implicit sentiment . they propose to train large-scale sentiment-annotated corpora from in-domain language resources . |
| Outcome: | The proposed model achieves state-of-the-art performance on SemEval2014 benchmarks and comprehensively validates its effectiveness on learning implicit sentiment. |
SKEP: Sentiment Knowledge Enhanced Pre-training for Sentiment Analysis (2020.acl-main)
Copied to clipboard
| Challenge: | sentiment knowledge is ignored in sentiment analysis, despite its use in pretraining. |
| Approach: | They propose to use sentiment knowledge to learn a unified sentiment representation for multiple sentiment analysis tasks. |
| Outcome: | The proposed method outperforms strong pre-training baseline on three kinds of sentiment tasks. |
What is the best recipe for character-level encoder-only modelling? (2023.acl-long)
Copied to clipboard
| Challenge: | aims to benchmark recent progress in language understanding models that output contextualised representations at the character level. |
| Approach: | They aim to find the best way to build and train character-level BERT-like models by comparing architectural innovations with pretraining objectives. |
| Outcome: | The proposed model outperforms a token-based model on a set of evaluation tasks with a fixed training procedure. |
CharBERT: Character-aware Pre-trained Language Model (2020.coling-main)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) construct word representations at subword level with Byte-Pair Encoding (BPE) or its variations . but these methods split a word into subword units and make it incomplete and fragile . |
| Approach: | They propose a character-aware pre-trained language model to tackle OOV problems . they construct contextual word embedding for each token from sequential character representations . |
| Outcome: | The proposed model improves on the existing models on multiple NLP benchmarks. |
Exploring Multilingual Pre-trained Language Model for Aspect-based Sentiment Analysis (2026.findings-acl)
Copied to clipboard
| Challenge: | Aspect-based sentiment analysis studies have focused on English datasets, but labeled data is scarce. |
| Approach: | They propose a multilingual pre-trained language model that leverages bilingual pre-training to leverage aspects-based sentiment analysis. |
| Outcome: | The proposed model outperforms state-of-the-art models across multiple languages. |
Encoding Sentiment Information into Word Vectors for Sentiment Analysis (C18-1)
Copied to clipboard
| Challenge: | Existing methods for embedding sentiment knowledge into word vectors are generally trained independently of the downstream task. |
| Approach: | They propose to encode sentiment knowledge into pre-trained word vectors to improve sentiment analysis. |
| Outcome: | The proposed method improves sentiment analysis on four popular sentiment datasets compared to benchmark methods. |
Vision-Language Pre-Training for Multimodal Aspect-Based Sentiment Analysis (2022.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to multimodal Aspect-Based Sentiment Analysis (MABSA) ignore crossmodalalignment and use pre-trained visual and textual models. |
| Approach: | They propose a multimodal multimodal encoder-decoder framework for MABSA that uses a unified multimodal decoder architecture for all the pretrainingand downstream tasks. |
| Outcome: | The proposed framework outperforms state-of-the-art approaches on three MABSA subtasks. |