| Challenge: | Using dice loss, we find that data imbalance is a common issue in many NLP tasks . data imbalance affects the performance of many tasks, such as tagging and machine reading comprehension . |
| Approach: | They propose to use dice loss to replace the standard cross-entropy objective for data-imbalanced NLP tasks. |
| Outcome: | The proposed training objective achieves significant performance boost on a wide range of data imbalanced tasks. |
Similar Papers
MiLe Loss: a New Loss for Mitigating the Bias of Learning Difficulties in Generative Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing generative language models neglect an inherent challenge in text corpus during training, i.e., the imbalance between frequent tokens and infrequent ones. |
| Approach: | They propose a function to mitigate the imbalance between frequent and infrequent tokens . authors propose 'MiLe Loss' function to assess learning difficulty of tokens during training . |
| Outcome: | Experiments show that models with proposed model can improve on downstream benchmarks. |
A Survey of Methods for Addressing Class Imbalance in Deep-Learning Based Natural Language Processing (2023.eacl-main)
Copied to clipboard
| Challenge: | Developing methods to improve model performance in imbalanced data settings has been an active area for decades . |
| Approach: | They propose to use sampling, data augmentation, choice of loss function, staged learning, or model design to address class imbalance in NLP. |
| Outcome: | The proposed approaches are evaluated on a variety of NLP tasks or in the computer vision community. |
Batch-Softmax Contrastive Loss for Pairwise Sentence Scoring Tasks (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent advances in machine learning have led to the use of contrastive loss for representation learning. |
| Approach: | They propose to use batch-softmax contrastive loss to train pairwise sentence embeddings . they propose to take a batch-softermax contrastitive loss and train it with different loss functions . |
| Outcome: | The proposed model improves on a number of datasets and pairwise sentence scoring tasks. |
Robust Lottery Tickets for Pre-trained Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that pre-trained language models contain smaller matching subnetworks that are not robust to adversarial examples. |
| Approach: | They propose a method to find robust tickets hidden in pre-trained language models by learning binary weight masks and an adversarial loss objective to guide the search. |
| Outcome: | The proposed method improves on previous work on adversarial robustness evaluation. |
Striking a Balance: Alleviating Inconsistency in Pre-trained Models for Symmetric Classification Tasks (2022.findings-acl)
Copied to clipboard
| Challenge: | Inconsistency is observed in symmetric classification tasks that take two inputs and require the output to be invariant of the order of the inputs. |
| Approach: | They propose a consistency loss function to alleviate inconsistency in symmetric classification tasks that take two inputs and require the output to be invariant of the order of the inputs. |
| Outcome: | The proposed model improves consistency in predictions for three paraphrase detection datasets without significant drop in accuracy scores. |
Calibrating Imbalanced Classifiers with Focal Loss: An Empirical Study (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Imbalanced data distributions can cause models to overfit to majority classes and output unreliable (mostly overconfident) predictions. |
| Approach: | They propose to streamline the model development and deployment using focal loss to address imbalanced data distributions. |
| Outcome: | The proposed model training with focal loss improves calibration and accuracy compared to standard cross-entropy loss. |
Fairness-aware Class Imbalanced Learning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on class imbalance and mitigating bias have focused on the latter . a skewed class distribution hurts the performance of deep learning models, and is often referred to as "stereotyping" |
| Approach: | They propose to extend a margin-loss based approach to enforce fairness by using tweet sentiment and occupation classification to mitigate class imbalance and demographic bias. |
| Outcome: | The proposed methods help mitigate class imbalance and demographic biases through controlled experiments. |
On the Importance of Adaptive Data Collection for Extremely Imbalanced Pairwise Tasks (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent datasets heuristically choose examples to ensure label balance . state-of-the-art models trained on QQP and WikiQA have only 2.4% average precision . |
| Approach: | They show that recent datasets heuristically choose examples to ensure label balance . they instead use active learning to retrieve uncertain points from a large pool of unlabeled utterance pairs . |
| Outcome: | The proposed model improves on QQP and WikiQA by using more informative negative examples. |
Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019) (D19-61)
Copied to clipboard
| Challenge: | EMNLP-IJCNLP 2019 Workshop on Deep Learning Approaches for Low-Resource Natural Language Processing takes place in Hong Kong, China . |
| Approach: | EMNLP-IJCNLP 2019 Workshop on Deep Learning Approaches for Low-Resource Natural Language Processing takes place in Hong Kong, China . call for papers for this second workshop met with a strong response . |
| Outcome: | the EMNLP-IJCNLP 2019 workshop on deep learning approaches for low-resource natural language processing takes place in Hong Kong, China. |
Improving Influence-based Instruction Tuning Data Selection for Balanced Learning of Diverse Capabilities (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Influence-based methods show promise in achieving (1), but often struggle with (2) . data selection is often biased towards high-influence tasks, harming performance on them . |
| Approach: | They propose a Balanced and Influential Data Selection algorithm that normalizes influence scores of training data and iteratively chooses the training example with the highest influence on the most underrepresented task. |
| Outcome: | The proposed model outperforms both state-of-the-art influence-based methods and non-influence-based frameworks on seven benchmarks spanning five diverse capabilities. |