Learning Label Hierarchy with Supervised Contrastive Learning (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing approaches to supervised contrastive learning treat each class as independent and therefore consider all classes to be equally important. |
| Approach: | They propose a family of Label-Aware SCL methods that incorporate hierarchical information to SCL by leveraging similarities between classes. |
| Outcome: | The proposed method outperforms baseline supervised approaches on three datasets. |
Similar Papers
Instances and Labels: Hierarchy-aware Joint Supervised Contrastive Learning for Hierarchical Multi-Label Text Classification (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to hierarchical multi-label text classification (HMTC) ignore the correlation between similar samples and introduce noise . |
| Approach: | They propose a semi-supervised method that uses a label hierarchy to bring text and label embeddings closer to each other by supervised contrastive learning. |
| Outcome: | The proposed method bridges the gap between supervised contrastive learning and HMTC by bringing text and label embeddings closer. |
Supporting Clustering with Contrastive Learning (2021.naacl-main)
Copied to clipboard
Dejiao Zhang, Feng Nan, Xiaokai Wei, Shang-Wen Li, Henghui Zhu, Kathleen McKeown, Ramesh Nallapati, Andrew O. Arnold, Bing Xiang
| Challenge: | Unsupervised clustering aims at discovering the semantic categories of data according to some distance measured in the representation space, but different categories overlap with each other at the beginning of the learning process. |
| Approach: | They propose a framework to leverage contrastive learning to promote better separation between different categories by optimizing a clustering objective defined in the representation space. |
| Outcome: | The proposed framework improves state-of-the-art accuracy and normalized mutual information on short text clustering and combines top-down and bottom-up instance discrimination to achieve better distances. |
Contrastive Learning-Enhanced Nearest Neighbor Mechanism for Multi-Label Text Classification (2022.acl-short)
Copied to clipboard
| Challenge: | Existing methods for multi-label text classification neglect the knowledge from the existing similar instances when predicting labels of a specific text. |
| Approach: | They propose a k nearest neighbor mechanism which retrieves several neighbor instances and interpolates the model output with their labels. |
| Outcome: | Extensive experiments show that the proposed method can bring significant performance improvements to multiple MLTC models including state-of-the-art pretrained and non-pretrained ones. |
Label Anchored Contrastive Learning for Language Understanding (2022.naacl-main)
Copied to clipboard
| Challenge: | a novel approach to contrastive learning for language understanding is not fully explored . contrastive training has been widely applied to self-supervised representation learning . |
| Approach: | They propose a label anchored contrastive learning approach for language understanding using a class label. |
| Outcome: | The proposed approach improves on GLUE and CLUE benchmarks by 4.1% compared to the state-of-the-art approaches . the proposed approach also improves under the few-shot and data imbalance settings . |
Incorporating Hierarchy into Text Encoder: a Contrastive Learning Approach for Hierarchical Text Classification (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods encode text and label hierarchy separately and mix their representations for classification, where the hierarchy remains unchanged for all input text. |
| Approach: | They propose to embed hierarchy into a text encoder by combining input and output data to generate a hierarchy-aware representation. |
| Outcome: | Extensive experiments on three benchmark datasets verify the effectiveness of the proposed model. |
An Effective Deployment of Contrastive Learning in Multi-label Text Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on contrastive learning in natural language processing tasks have not explored the effectiveness of the technology. |
| Approach: | They propose five novel contrastive losses for multi-label text classification tasks that exploit the complexity of the input logic and the semantic representation space. |
| Outcome: | The proposed contrastive losses improve multi-label text classification tasks and can be adapted for multi-task tasks. |
HILL: Hierarchy-aware Information Lossless Contrastive Learning for Hierarchical Text Classification (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing self-supervised methods in natural language processing rely on augmentation rules to generate contrastive samples. |
| Approach: | They propose a hierarchy-aware information lossless contrastive learning scheme that uses syntactic information reserved in the input sample and fused during the learning process. |
| Outcome: | The proposed learning scheme is superior to existing methods in hierarchical text classification . the proposed learning system is based on a structure encoder and a text encoder . |
Keywords and Instances: A Hierarchical Contrastive Learning Framework Unifying Hybrid Granularities for Text Generation (2022.acl-long)
Copied to clipboard
Mingzhe Li, XieXiong Lin, Xiuying Chen, Jinxiong Chang, Qishen Zhang, Feng Wang, Taifeng Wang, Zhongyi Liu, Wei Chu, Dongyan Zhao, Rui Yan
| Challenge: | Existing studies focus on contrastive learning on the instance level without discriminating the contribution of each word. |
| Approach: | They propose a hierarchical contrastive learning mechanism which can unify semantic meaning in the input text. |
| Outcome: | The proposed model outperforms baselines on storytelling, paraphrasing, dialogue generation, and storytelling tasks. |
Tighter Clusters, Safer Code? Improving Vulnerability Detection with Enhanced Contrastive Loss (2025.naacl-srw)
Copied to clipboard
| Challenge: | a recent study shows that supervised contrastive learning (SCL) improves embedding separation but struggles with intra-class clustering . code vulnerability detection is a cornerstone of software security, especially as the world undergoes rapid digitization. |
| Approach: | They propose an extension of SCL with a distance-based regularization term that tightens intra-class clustering while maintaining inter-class separation. |
| Outcome: | The proposed method improves F1 score on CodeBERT and GraphCodeBERT with BCE, BCE + SCL, and BCE+ CESCL. |
TLSA: LLM-Guided Text-Label Space Alignment with Contrastive Learning for Generalized Category Discovery (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for generalized category discovery suffer from weak text–label alignment, inconsistent objectives across known and novel categories, and poor discrimination of semantically similar clusters. |
| Approach: | They propose a unified framework that enforces contrastive alignment between text and label representations within a shared semantic space. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on four benchmark datasets. |