Instance-Selection-Inspired Undersampling Strategies for Bias Reduction in Small and Large Language Models for Binary Text Classification (2025.acl-long)
Copied to clipboard
Guilherme Fonseca, Washington Cunha, Gabriel Prenassi, Marcos André Gonçalves, Leonardo Chaves Dutra Da Rocha
| Challenge: | Existing methods to mitigate class imbalanced datasets are limited by existing methods. |
| Approach: | They propose two undersampling methods inspired by state-of-the-art Instance Selection techniques to mitigate class imbalance bias in ATC. |
| Outcome: | The proposed methods reduce classifier bias (56%) across all datasets without effectiveness loss while improving efficiency (1.6x speedup), scalability and reducing carbon emissions (up to 50%). |
Similar Papers
Investigating Ensemble Methods for Model Robustness Improvement of Text Classifiers (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to reduce model's reliance on bias features ignore the learnability of these features. |
| Approach: | They propose to reduce models' reliance on bias features by first training models with fixed low-capacity models which ignore the learnability of the bias features. |
| Outcome: | The proposed models can perform better on out-of-distribution datasets than baseline models with a more sophisticated model design. |
A Survey of Methods for Addressing Class Imbalance in Deep-Learning Based Natural Language Processing (2023.eacl-main)
Copied to clipboard
| Challenge: | Developing methods to improve model performance in imbalanced data settings has been an active area for decades . |
| Approach: | They propose to use sampling, data augmentation, choice of loss function, staged learning, or model design to address class imbalance in NLP. |
| Outcome: | The proposed approaches are evaluated on a variety of NLP tasks or in the computer vision community. |
Likelihood-based Mitigation of Evaluation Bias in Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are widely used to evaluate natural language generation tasks as automated metrics. |
| Approach: | They propose to use LLMs to evaluate sentences with higher likelihoods and lower likelihoods to mitigate the likelihood bias. |
| Outcome: | The proposed method overrates sentences with higher likelihoods while underrating sentences with lower likelihoods. |
Demographics Should Not Be the Reason of Toxicity: Mitigating Discrimination in Text Classifications with Instance Weighting (2020.acl-main)
Copied to clipboard
| Challenge: | Recent research has found that text classification datasets contain certain unintended biases, such as text containing demographic identity-terms that are more likely to be abusive. |
| Approach: | They propose a model-agnostic debiasing framework that recovers the non-discrimination distribution using instance weighting, which does not require extra resources or annotations apart from a pre-defined set of demographic identity-terms. |
| Outcome: | The proposed framework alleviates the unintended biases without hurting models’ generalization ability. |
Rethinking Pruning Large Language Models: Benefits and Pitfalls of Reconstruction Error Minimization (2024.emnlp-main)
Copied to clipboard
| Challenge: | minimizing reconstruction error is not always ideal and can overfit calibration data. |
| Approach: | They propose a method to prune large language models by divide and conquer . they propose minimizing reconstruction error by more than 90% by using calibration data . |
| Outcome: | The proposed pruning approach generates high reconstruction errors . the proposed technique reduces reconstruction error by more than 90% . |
Large Language Models Are Still Misled by Simple Bias Ensembles (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for large language models are constrained to datasets where each sample is manually injected with only one type of bias. |
| Approach: | They propose a multi-bias benchmark where each sample contains multiple types of biases. |
| Outcome: | The proposed benchmark shows that existing LLMs and debiasing methods perform poorly on this benchmark, highlighting the challenge of eliminating compounded biases. |
Methods for Estimating and Improving Robustness of Language Models (2022.naacl-srw)
Copied to clipboard
| Challenge: | Large language models suffer from weak generalisation ability due to shallow textual relations over full semantic complexity of the problem. |
| Approach: | They propose to incorporate some of these measures into training objectives to enhance distributional robustness of LLMs. |
| Outcome: | The proposed models outperform human models on complex tasks and outperformed other models on deep networks. |
Debiasing Large Language Models with Structured Knowledge (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to reduce biases in pre-training models are hampered by their performance. |
| Approach: | They propose a method that utilizes structured knowledge to mitigate bias in LLMs . their method obviates the need for training from scratch, thus offering enhanced scalability . |
| Outcome: | The proposed method outperforms state-of-the-art (SOTA) baselines in the debiasing ability. |
Mitigating Boundary Ambiguity and Inherent Bias for Text Classification in the Era of Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | a new text classification framework for large language models addresses the problem of boundary ambiguity and inherent biases in LLMs. |
| Approach: | They propose a two-stage classification framework for large language models to mitigate bottlenecks . their approach uses pairwise comparisons to efficiently narrow down options . |
| Outcome: | The proposed framework reduces the number of options and improves on four datasets. |
On the Limitations of Dataset Balancing: The Lost Battle Against Spurious Correlations (2022.findings-naacl)
Copied to clipboard
| Challenge: | Recent work shows that deep learning models are sensitive to low-level correlations between simple features and specific output labels, leading to over-fitting and lack of generalization. |
| Approach: | They propose to eliminate single-word correlations altogether to mitigate this problem . they highlight several alternatives to dataset balancing to enhance contexts . |
| Outcome: | The proposed approach to balancing datasets is insufficient, the authors argue . they suggest enhancing datasets with richer contexts and abstaining from interaction . |