Learning on Imbalanced Noisy Data via Debiased Sample Selection and LLM-Driven Annotation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to learning with noisy labels are prone to selection bias and training bias . obtaining large-scale high-quality datasets is expensive and time-consuming in practical scenarios . |
| Approach: | They propose an imbalanced learning with noisy labels task to let model learn from noisy labels . they first conduct debiased sample selection to better separate clean samples from noisy samples . then they feed selected clean samples to active annotator large language models for re-annotating noisy samples. |
| Outcome: | The proposed method is superior to existing methods on synthetic and real-world datasets. |
Similar Papers
Hide and Seek in Noise Labels: Noise-Robust Collaborative Active Learning with LLMs-Powered Assistance (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for learning from noisy labels are difficult to improve . existing methods identify noisy labels and use active learning to query experts . |
| Approach: | They propose a collaborative learning framework to combine LLMs and small models for learning from noisy labels. |
| Outcome: | The proposed framework outperforms state-of-the-art baselines on synthetic and real-world noise datasets. |
NoisywikiHow: A Benchmark for Learning with Real-world Noisy Labels in Natural Language Processing (2023.findings-acl)
Copied to clipboard
| Challenge: | Large-scale datasets in the real world often contain label noise, which can cause model overfitting and degrade generalization. |
| Approach: | They propose to use label noise to imitate human errors in annotations . they use a noisy label noise benchmark to evaluate their methods . |
| Outcome: | The proposed benchmarks are different from data with heterogeneous label noises in the real world. |
The Promises and Pitfalls of LLM Annotations in Dataset Labeling: a Case Study on Media Bias Detection (2025.findings-naacl)
Copied to clipboard
Tomáš Horych, Christoph Mandl, Terry Ruas, Andre Greiner-Petter, Bela Gipp, Akiko Aizawa, Timo Spinde
| Challenge: | Recent research suggests using Large Language Models (LLMs) to automate the annotation process, reducing these costs while maintaining data quality. |
| Approach: | They propose to use Large Language Models to automate annotation process and train classifiers on large datasets. |
| Outcome: | The proposed model outperforms all of the annotator LLMs on two media bias benchmark datasets (BABE and BASIL) while maintaining data quality. |
Learning with Less: Knowledge Distillation from Large Language Models via Unlabeled Data (2025.findings-naacl)
Copied to clipboard
Juanhui Li, Sreyashi Nag, Hui Liu, Xianfeng Tang, Sheikh Muhammad Sarwar, Limeng Cui, Hansu Gu, Suhang Wang, Qi He, Jiliang Tang
| Challenge: | Large Language Models (LLMs) have demonstrated superior language understanding abilities in many real-world NLP applications. |
| Approach: | They propose a learning-based sample selection method that incorporates signals from both teacher and student to enhance model performance. |
| Outcome: | The proposed method improves model performance across datasets with higher data efficiency. |
Weed Out, Then Harvest: Dual Low-Rank Adaptation is an Effective Noisy Label Detector for Noise-Robust Learning (2025.findings-acl)
Copied to clipboard
| Challenge: | Experimental results show that PEFT can fine-tune language models without relying on perfectly labeled datasets. |
| Approach: | They propose a framework that decouples sample selection from model training by introducing clean and noisy LoRA. |
| Outcome: | The proposed framework decouples sample selection from model training. |
Dual Debiasing for Noisy In-Context Learning for Text Generation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods detect noisy annotations by ranking local perplexities, but this assumption breaks down when the noise ratio is high and many demonstrations are flawed. |
| Approach: | They propose a method that uses synthesized neighbors to explicitly correct perplexity estimates, yielding a robust Sample Cleanliness Score. |
| Outcome: | The proposed method is comparable to a fully clean demonstration corpus and performs well even with noise ratios as high as 0.8. |
Adaptive Ranking-based Sample Selection for Weakly Supervised Class-imbalanced Text Classification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to synthesize training labels with labeling rules ignore data imbalance issue . weak supervision paradigm is often used to reduce human efforts to produce training labels inexpensively. |
| Approach: | They propose a model-agnostic framework to alleviate the data imbalance issue in the weak supervision paradigm by combining labeling rules with a probabilistic margin score. |
| Outcome: | The proposed framework outperforms the state-of-the-art imbalanced learning and WS methods on four text classification datasets with four different imbalance ratios. |
Instance-Selection-Inspired Undersampling Strategies for Bias Reduction in Small and Large Language Models for Binary Text Classification (2025.acl-long)
Copied to clipboard
Guilherme Fonseca, Washington Cunha, Gabriel Prenassi, Marcos André Gonçalves, Leonardo Chaves Dutra Da Rocha
| Challenge: | Existing methods to mitigate class imbalanced datasets are limited by existing methods. |
| Approach: | They propose two undersampling methods inspired by state-of-the-art Instance Selection techniques to mitigate class imbalance bias in ATC. |
| Outcome: | The proposed methods reduce classifier bias (56%) across all datasets without effectiveness loss while improving efficiency (1.6x speedup), scalability and reducing carbon emissions (up to 50%). |
Benchmarking Debiasing Methods for LLM-based Parameter Estimates (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are expensive yet powerful ways to annotate text, and can be inconsistent when compared with experts. |
| Approach: | They propose to combine LLM annotations with a limited number of expensive expert annotations to produce valid estimates. |
| Outcome: | The proposed methods produce consistent estimates under theoretical assumptions, but they are not comparable across finite datasets. |
Are LLMs Better than Reported? Detecting Label Errors and Mitigating Their Effect on Model Performance (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) offer new opportunities to enhance the annotation process, particularly for detecting label errors in existing datasets. |
| Approach: | They propose to use an ensemble of large language models to flag mislabeled examples by using an LLM-as-a-judge approach to detect label errors in existing datasets. |
| Outcome: | The proposed method improves label accuracy and consistency in large language models. |