Self-Training for Sample-Efficient Active Learning for Text Classification with Pre-Trained Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to train models without labeled data are lacking in supervised tasks . a lack of labeles is the main obstacle to real-world applications . |
| Approach: | They propose a semi-supervised approach that uses a model to obtain pseudo-labels for unlabeled data. |
| Outcome: | The proposed method outperforms the reproduced methods on four text classification benchmarks. |
Similar Papers
Leveraging Training Dynamics and Self-Training for Text Classification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Semi-supervised learning (SSL) is a promising technique for improving deep learning models when training data is scarce. |
| Approach: | They propose a semi-supervised learning approach that leverages training dynamics of unlabeled data. |
| Outcome: | The proposed method achieves an average increase in F1 score of 3.5% over baselines in low resource settings. |
Active Learning for BERT: An Empirical Study (2020.emnlp-main)
Copied to clipboard
Liat Ein-Dor, Alon Halfon, Ariel Gera, Eyal Shnarch, Lena Dankin, Leshem Choshen, Marina Danilevsky, Ranit Aharonov, Yoav Katz, Noam Slonim
| Challenge: | Existing approaches to deal with data scarcity are active learning (AL) and pre-trained models are not being considered. |
| Approach: | They propose to use active learning techniques to cope with data scarcity in binary text classification scenarios where the annotation budget is very small and the data is often skewed. |
| Outcome: | The proposed methods improve BERT performance in binary text classification scenarios where the annotation budget is very small and the data is often skewed. |
Cold-start Active Learning through Self-supervised Language Modeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Labeling data is a fundamental bottleneck in machine learning due to annotation cost and time. |
| Approach: | They propose a strategy that uses the pre-training loss to find examples that surprise the model and minimize labeling costs. |
| Outcome: | The proposed approach reduces labeling costs and costs by using pre-trained language models. |
Data-efficient Active Learning for Structured Prediction with Partial Annotation and Self-Training (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Structured prediction is a fundamental problem in NLP, wherein the label space consists of complex structured outputs with groups of interdependent variables. |
| Approach: | They propose a partial annotation approach that selects only the most informative sub-structures for annotation and a method that incorporates the current model's automatic predictions as pseudo-labels for un-annotated sub-structurals. |
| Outcome: | The proposed approach reduces annotation cost over strong full annotation baselines under a fair comparison scheme that takes reading time into consideration. |
Self-training Improves Pre-training for Natural Language Understanding (2021.naacl-main)
Copied to clipboard
Jingfei Du, Edouard Grave, Beliz Gunel, Vishrav Chaudhary, Onur Celebi, Michael Auli, Veselin Stoyanov, Alexis Conneau
| Challenge: | Unsupervised pretraining has led to improvements in natural language understanding . a data augmentation method can be used to generate labels for unlabeled examples . |
| Approach: | They propose a semi-supervised method which uses unlabeled data to retrieve sentences from a database of billions of unlabed sentences crawled from the web. |
| Outcome: | The proposed method improves on standard text classification benchmarks by 2.6% and knowledge distillation by few shots. |
Reinforced Active Learning for Low-Resource, Domain-Specific, Multi-Label Text Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | Modern text classification systems achieve excellent accuracy across tasks and corpora. |
| Approach: | They propose a Reinforcement Learning policy that uses many different aspects of the data and task to select the most informative unlabeled subset dynamically over the course of the AL procedure. |
| Outcome: | The proposed framework outperforms baselines on four complex multi-class, multi-label text classification datasets. |
Neural Networks Against (and For) Self-Training: Classification with Small Labeled and Large Unlabeled Sets (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models for text classification suffer from the semantic drift problem, which is a problem for self-training. |
| Approach: | They propose a semi-supervised text classifier based on self-training using one positive and one negative property of neural networks. |
| Outcome: | The proposed model outperforms ten baseline models in five benchmarks and is additive to language model pretraining. |
Investigating Active Learning Sampling Strategies for Extreme Multi Label Text Classification (2022.lrec-1)
Copied to clipboard
| Challenge: | Large scale, multi-label text datasets with high numbers of different classes are expensive to annotate due to domain experts taking a lot of time working through all the classes. |
| Approach: | They propose to build classifiers on multi-label text datasets using Active Learning to reduce labeling effort. |
| Outcome: | The proposed classifiers can be used to reduce labeling effort on multi-label datasets. |
LLMaAA: Making Large Language Models as Active Annotators (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing supervised learning methods in natural language processing require large amounts of data. |
| Approach: | They propose an active learning loop that takes LLMs as annotators and puts them into an active loop to determine what to annotate efficiently. |
| Outcome: | The proposed model outperforms existing models with few-shot performance in two NLP tasks. |
Text Classification Using Label Names Only: A Language Model Self-Training Approach (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current text classification methods require a large number of labeled documents as training data. |
| Approach: | They propose a model that uses only the label name of each class to train classification models on unlabeled data without using any labeled examples. |
| Outcome: | The proposed model achieves 90% accuracy on four benchmark datasets using label names as the only supervision . |