| Challenge: | Existing studies on active learning identify sampling bias in large datasets . cost and time needed for labeling and model training are bottlenecks preventing new and/or better models from being trained . |
| Approach: | They propose to use active learning to identify representative data samples for training . they propose to create tiny datasets that can be used for cheap training if needed . |
| Outcome: | The proposed model outperforms the state-of-the-art on active text classification using small representative datasets with active learning. |
Similar Papers
Active Learning for BERT: An Empirical Study (2020.emnlp-main)
Copied to clipboard
Liat Ein-Dor, Alon Halfon, Ariel Gera, Eyal Shnarch, Lena Dankin, Leshem Choshen, Marina Danilevsky, Ranit Aharonov, Yoav Katz, Noam Slonim
| Challenge: | Existing approaches to deal with data scarcity are active learning (AL) and pre-trained models are not being considered. |
| Approach: | They propose to use active learning techniques to cope with data scarcity in binary text classification scenarios where the annotation budget is very small and the data is often skewed. |
| Outcome: | The proposed methods improve BERT performance in binary text classification scenarios where the annotation budget is very small and the data is often skewed. |
Deep Bayesian Active Learning for Natural Language Processing: Results of a Large-Scale Empirical Study (D18-1)
Copied to clipboard
| Challenge: | Existing studies on Active Learning (AL) for natural language processing have limited data requirements. |
| Approach: | They propose a Bayesian active learning approach that reduces deep learning's data dependence by comparing models and acquisition functions. |
| Outcome: | The proposed approach outperforms i.i.d. baselines and is more efficient than other approaches. |
Investigating Multi-source Active Learning for Natural Language Inference (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent studies often assume that training and test data are drawn from the same distribution. |
| Approach: | They propose to apply active learning to unlabelled data pools to test for learning and generalisation. |
| Outcome: | The proposed strategies outperform random selection and outperformed hard-to-learn data on the task of natural language inference. |
Investigating Active Learning Sampling Strategies for Extreme Multi Label Text Classification (2022.lrec-1)
Copied to clipboard
| Challenge: | Large scale, multi-label text datasets with high numbers of different classes are expensive to annotate due to domain experts taking a lot of time working through all the classes. |
| Approach: | They propose to build classifiers on multi-label text datasets using Active Learning to reduce labeling effort. |
| Outcome: | The proposed classifiers can be used to reduce labeling effort on multi-label datasets. |
Active Sentence Learning by Adversarial Uncertainty Sampling in Discrete Space (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing uncertainty sampling methods are time-consuming and can't be executed frequently. |
| Approach: | They propose adversarial uncertainty sampling in discrete space to find informative unlabeled text samples for annotation using adversarials. |
| Outcome: | The proposed approach outperforms baselines on effectiveness on five datasets. |
Reassessing Active Learning Adoption in Contemporary NLP: A Community Survey (2026.eacl-long)
Copied to clipboard
| Challenge: | a longstanding strategy to reduce annotation costs is active learning . data annotation is expected to remain important and active learning to stay relevant . |
| Approach: | They conduct an online survey to assess the perceived relevance of data annotation and active learning . they propose a strategy to reduce annotation costs using active learning, an iterative process . |
| Outcome: | The proposed strategies reduce setup complexity and uncertainty cost while maintaining model performance. |
D-CALM: A Dynamic Clustering-based Active Learning Approach for Mitigating Bias (2023.findings-acl)
Copied to clipboard
| Challenge: | Infusing clustering with active learning with AL can overcome the bias issue of both AL and traditional annotation methods while exploiting AL’s annotation efficiency. |
| Approach: | They propose an algorithm that dynamically adjusts clustering and annotation efforts in response to an estimated classifier error-rate. |
| Outcome: | The proposed algorithm outperforms baseline AL approaches with pretrained transformers and traditional Support Vector Machines on eight datasets for emotion, hatespeech, dialog act, and book type detection tasks. |
On the Fragility of Active Learners for Text Classification (2024.emnlp-main)
Copied to clipboard
| Challenge: | Active learning (AL) techniques optimally utilize a labeling budget by iteratively selecting instances that are most valuable for learning. |
| Approach: | They propose to use active learning techniques to iteratively select instances that are most valuable for learning. |
| Outcome: | The proposed framework is used to benchmark active learning techniques for text classification using pre-trained representations. |
Self-Training for Sample-Efficient Active Learning for Text Classification with Pre-Trained Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to train models without labeled data are lacking in supervised tasks . a lack of labeles is the main obstacle to real-world applications . |
| Approach: | They propose a semi-supervised approach that uses a model to obtain pseudo-labels for unlabeled data. |
| Outcome: | The proposed method outperforms the reproduced methods on four text classification benchmarks. |
On the Limitations of Simulating Active Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Active learning (AL) is a human-and-model-in-the-loop paradigm that iteratively selects informative unlabeled data for human annotation. |
| Approach: | They propose to simulate active learning by using an already labeled dataset as the pool of unlabeled data. |
| Outcome: | The proposed model-in-the-loop paradigm can be used to perform experiments with human annotations on-the fly. |