SoftMCL: Soft Momentum Contrastive Learning for Fine-grained Sentiment-aware Pre-training (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for pre-training language models capture general language understanding but fail to distinguish affective impact of a particular context to a specific word. |
| Approach: | They propose a soft momentum contrastive learning method for fine-grained sentiment-aware pre-training that uses valence ratings as soft-label supervision instead of hard labels. |
| Outcome: | The proposed method improves on four sentiment-related tasks and the results are published online. |
Similar Papers
Attentive Gated Lexicon Reader with Contrastive Contextual Co-Attention for Sentiment Classification (D18-1)
Copied to clipboard
| Challenge: | Existing sentiment lexicons do not handle word sense and the concept of semantic compositionality is non-existent in simple lexiconic approaches. |
| Approach: | They propose a lexicon-driven contextual attention mechanism and a contrastive co-attention mechanism that models contrasting polarities between all positive and negative words in a sentence. |
| Outcome: | The proposed model outperforms many other neural baselines on sentiment classification tasks on multiple benchmark datasets. |
Not All Negatives are Equal: Label-Aware Contrastive Loss for Fine-grained Text Classification (2021.emnlp-main)
Copied to clipboard
| Challenge: | Fine-grained classification tasks involve distinguishing between classes with subtle differences between them. |
| Approach: | They analyse fine-grained text classification tasks by embedding class relationships into a contrastive objective function to help differently weigh the positives and negatives. |
| Outcome: | The proposed model outperforms previous contrastive methods on emotion classification and sentiment analysis. |
Retrofitting Light-weight Language Models for Emotions using Supervised Contrastive Learning (2023.emnlp-main)
Copied to clipboard
| Challenge: | a novel retrofitting method to induce emotion aspects into pre-trained language models is proposed . the models are computationally less expensive and open, but do not capture affective aspects of human communication well. |
| Approach: | They propose a retrofitting method to induce emotion aspects into pre-trained language models . they retrofit text fragments exhibiting similar emotions into pretrained networks . |
| Outcome: | The proposed method produces emotion-aware text representations for sentiment analysis and sarcasm detection tasks. |
CLOWER: A Pre-trained Language Model with Contrastive Learning over Word and Character Representations (2022.coling-1)
Copied to clipboard
Borun Chen, Hongyin Tang, Jiahao Bu, Kai Zhang, Jingang Wang, Qifan Wang, Hai-Tao Zheng, Wei Wu, Liqian Yu
| Challenge: | Pre-trained language models (PLMs) have achieved remarkable performance gains across numerous downstream tasks in natural language understanding. |
| Approach: | They propose a Chinese pre-trained language model that implicitly encodes words into characters . they propose 'contrastive learning over word' and 'character' representations to improve learning . |
| Outcome: | The proposed model can encode words into fine-grained representations without modification of production pipelines. |
Soft-Labeled Contrastive Pre-Training for Function-Level Code Representation (2022.findings-emnlp)
Copied to clipboard
Xiaonan Li, Daya Guo, Yeyun Gong, Yun Lin, Yelong Shen, Xipeng Qiu, Daxin Jiang, Weizhu Chen, Nan Duan
| Challenge: | Existing methods for contrastive pre-training ignore the relevance between codes in large code corpus. |
| Approach: | They propose a Soft-labeled contrastive pre-training framework with positive sample construction methods to learn functional-level code representation. |
| Outcome: | The proposed framework can obtain fine-grained soft-labels through an iterative adversarial manner and use them to learn better code representation. |
A Triple-View Framework for Fine-Grained Emotion Classification with Clustering-Guided Contrastive Learning (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies have focused on dealing with only one of the two difficulties of coarse-grained emotion classification. |
| Approach: | They propose a triple-view framework that treats FEC as an instance-label joint embedding learning problem to tackle both difficulties concurrently by considering three complementary views. |
| Outcome: | The proposed framework achieves significant and consistent improvements on two widely-used benchmark datasets. |
Unsupervised Dense Retrieval with Relevance-Aware Contrastive Pre-Training (2023.findings-acl)
Copied to clipboard
| Challenge: | Dense retrievers have impressive performance, but their demand for abundant training data limits their application scenarios. |
| Approach: | They propose a method which uses unlabeled data to construct pseudo-positive examples from unlabelled data and then contrastively weighs the contrastive loss of different pairs according to the estimated relevance. |
| Outcome: | The proposed method beats the SOTA unsupervised Contriever model on BEIR and open-domain QA retrieval benchmarks and is a good few-shot learner. |
Mere Contrastive Learning for Cross-Domain Sentiment Analysis (2022.coling-1)
Copied to clipboard
| Challenge: | Existing approaches to cross-domain sentiment analysis are labor-intensive and time-consuming. |
| Approach: | They propose a modified contrastive objective with in-batch negative samples to allow sentence representations from the same class to be pushed closer while those from the different classes become further apart in the latent space. |
| Outcome: | The proposed model can achieve state-of-the-art in cross-domain and multi-domain sentiment analysis tasks while transferring knowledge learned in the source domain to the target domain. |
Label-aware Hard Negative Sampling Strategies with Momentum Contrastive Learning for Implicit Hate Speech Detection (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing models for implicit hate speech detection do not have significant advantage over cross-entropy loss-based learning. |
| Approach: | They propose a label-aware hard negative sampling strategy that encourages the model to learn detailed features from hard negative samples instead of random batch. |
| Outcome: | The proposed models outperform existing models for implicit hate speech detection both in- and cross-datasets. |
CLAPSpeech: Learning Prosody from Text Context with Contrastive Language-Audio Pre-Training (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for expressive text-to-speech only implicitly learn prosody with masked token reconstruction tasks. |
| Approach: | They propose a cross-modal contrastive pre-training framework that learns from prosody variance of the same text token under different contexts. |
| Outcome: | The proposed framework can learn from prosody variance of a text token under different contexts. |