Weighted Contrastive Learning With False Negative Control to Help Long-tailed Product Classification (2023.acl-industry)
Copied to clipboard
| Challenge: | Item categorization (IC) aims to classify a product into leaf nodes in a categorical taxonomy due to scarce supervision. |
| Approach: | They propose to use K-positive contrastive loss (KCL) to address IC task’s long-tail issue by re-weighting positive pairs in the KCL loss with a regularization that the sum of weights should be constrained to K+1 as close as possible. |
| Outcome: | The proposed method improves on the long-tail issue in the image classification task and when using text-based contrastive learning, it can be applied on the IC task. |
Similar Papers
An Effective Deployment of Contrastive Learning in Multi-label Text Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on contrastive learning in natural language processing tasks have not explored the effectiveness of the technology. |
| Approach: | They propose five novel contrastive losses for multi-label text classification tasks that exploit the complexity of the input logic and the semantic representation space. |
| Outcome: | The proposed contrastive losses improve multi-label text classification tasks and can be adapted for multi-task tasks. |
Not All Negatives are Equal: Label-Aware Contrastive Loss for Fine-grained Text Classification (2021.emnlp-main)
Copied to clipboard
| Challenge: | Fine-grained classification tasks involve distinguishing between classes with subtle differences between them. |
| Approach: | They analyse fine-grained text classification tasks by embedding class relationships into a contrastive objective function to help differently weigh the positives and negatives. |
| Outcome: | The proposed model outperforms previous contrastive methods on emotion classification and sentiment analysis. |
Debiased Contrastive Learning of Unsupervised Sentence Representations (2022.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that contrastive learning improves pre-trained language models to derive high-quality sentence representations. |
| Approach: | They propose a framework to punish false negatives and generate noise-based negatives to guarantee the uniformity of the representation space. |
| Outcome: | The proposed framework improves pre-trained language models while pushing apart irrelevant negatives to guarantee the uniformity of the representation space. |
Contrastive Learning-Enhanced Nearest Neighbor Mechanism for Multi-Label Text Classification (2022.acl-short)
Copied to clipboard
| Challenge: | Existing methods for multi-label text classification neglect the knowledge from the existing similar instances when predicting labels of a specific text. |
| Approach: | They propose a k nearest neighbor mechanism which retrieves several neighbor instances and interpolates the model output with their labels. |
| Outcome: | Extensive experiments show that the proposed method can bring significant performance improvements to multiple MLTC models including state-of-the-art pretrained and non-pretrained ones. |
Enhancing Input-Label Mapping in In-Context Learning with Contrastive Decoding (2025.acl-short)
Copied to clipboard
| Challenge: | Prior research has found that large language models overlook input-label mapping information in ICL, relying more on their pre-trained knowledge. |
| Approach: | They propose a novel method that contrasts input-label mappings between positive and negative in-context examples to improve model performance. |
| Outcome: | The proposed method improves performance on 7 natural language understanding tasks without additional training. |
Rethinking Negative Pairs in Code Search (2023.emnlp-main)
Copied to clipboard
| Challenge: | Comparative learning is a key component in fine-tuning code search models . however, negative samples of InfoNCE may deteriorate its representation learning . |
| Approach: | They propose a loss function that inserts weight terms into InfoNCE to improve contrastive learning. |
| Outcome: | The proposed loss function is a special case of Soft-InfoNCE, the authors show . it is more accurate than other loss functions, and it is faster than other models. |
SimKGC: Simple Contrastive Knowledge Graph Completion with Pre-trained Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Text-based methods lag behind graph embedding-based approaches for knowledge graph completion (KGC) |
| Approach: | They propose three types of negatives to improve contrastive learning to improve learning efficiency. |
| Outcome: | The proposed model outperforms embedding-based methods on several benchmark datasets. |
Balancing Methods for Multi-label Text Classification with Long-Tailed Class Distribution (2021.emnlp-main)
Copied to clipboard
| Challenge: | Multi-label text classification is a challenging task because it requires capturing label dependencies. |
| Approach: | They propose to use distribution-balanced loss functions to solve label dependency problems in multi-label text classification by capturing label dependencies from a fixed-set of labels. |
| Outcome: | The proposed loss function addresses both the class imbalance and label linkage problems and outperforms other loss functions. |
Clustering-Aware Negative Sampling for Unsupervised Sentence Representation (2023.findings-acl)
Copied to clipboard
| Challenge: | Using clustering-aware learning, in-batch negatives are often ignored in sentence representation learning. |
| Approach: | They propose a method that integrates cluster information into contrastive learning for unsupervised sentence representation learning. |
| Outcome: | The proposed method compares favorably with baselines on semantic textual similarity tasks. |
Improving Clustering with Positive Pairs Generated from LLM-Driven Labels (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing unsupervised clustering methods lack label knowledge, resulting in suboptimal performance. |
| Approach: | They propose to use LLM-driven labels to generate positive pairs from embedded data and an embedder to obviate the need for negative pairs. |
| Outcome: | The proposed framework surpasses state-of-the-art benchmarks on a range of datasets and generates interpretable labels for improved understanding of clustering results. |