Leveraging Training Dynamics and Self-Training for Text Classification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Semi-supervised learning (SSL) is a promising technique for improving deep learning models when training data is scarce. |
| Approach: | They propose a semi-supervised learning approach that leverages training dynamics of unlabeled data. |
| Outcome: | The proposed method achieves an average increase in F1 score of 3.5% over baselines in low resource settings. |
Similar Papers
Delta-training: Simple Semi-Supervised Text Classification using Pretrained Word Embeddings (D19-1)
Copied to clipboard
| Challenge: | Pretrained word embeddings outperforms classifiers with randomly initialized word embeds, a new method is proposed for semi-supervised text classification. |
| Approach: | They propose a method that uses pretrained word embeddings to predict text classification . they use unlabeled data to build a classifier, and use early-stopping to improve performance . |
| Outcome: | The proposed method outperforms self-training and co-training frameworks on unlabeled data. |
Neural Networks Against (and For) Self-Training: Classification with Small Labeled and Large Unlabeled Sets (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models for text classification suffer from the semantic drift problem, which is a problem for self-training. |
| Approach: | They propose a semi-supervised text classifier based on self-training using one positive and one negative property of neural networks. |
| Outcome: | The proposed model outperforms ten baseline models in five benchmarks and is additive to language model pretraining. |
Rethinking Semi-supervised Learning with Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Semi-supervised learning (SSL) is a popular setting to make use of unlabelled data . Currently, there are two popular approaches to make effective use of the unlabelled datasets . |
| Approach: | They compare semi-supervised learning (SSL) and task-adaptive pre-training (TAPT) they find TAPT is a stronger and more robust SSL learner, even when using just a few hundred unlabelled samples . |
| Outcome: | The proposed methods improve model performance across different NLP tasks and data sizes. |
Combining Deep Generative Models and Multi-lingual Pretraining for Semi-supervised Document Classification (2021.eacl-main)
Copied to clipboard
| Challenge: | Semi-supervised learning and multilingual pretraining have been shown to be effective for task-specific labelled data shortages. |
| Approach: | They propose to combine semi-supervised deep generative models and multi-lingual pretraining to form a pipeline for document classification task. |
| Outcome: | The proposed method outperforms state-of-the-art models in low-resource settings across several languages and outperformed existing models in English. |
Self-Training for Sample-Efficient Active Learning for Text Classification with Pre-Trained Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to train models without labeled data are lacking in supervised tasks . a lack of labeles is the main obstacle to real-world applications . |
| Approach: | They propose a semi-supervised approach that uses a model to obtain pseudo-labels for unlabeled data. |
| Outcome: | The proposed method outperforms the reproduced methods on four text classification benchmarks. |
Self-supervised Regularization for Text Classification (2021.tacl-1)
Copied to clipboard
| Challenge: | Text classification models are prone to overfitting when limited texts are available for training. |
| Approach: | They propose a data-dependent regularization approach based on self-supervised learning . they define auxiliary tasks on input data without using human-provided labels . |
| Outcome: | Experiments on 17 text classification datasets demonstrate the effectiveness of the proposed method. |
Text Classification Using Label Names Only: A Language Model Self-Training Approach (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current text classification methods require a large number of labeled documents as training data. |
| Approach: | They propose a model that uses only the label name of each class to train classification models on unlabeled data without using any labeled examples. |
| Outcome: | The proposed model achieves 90% accuracy on four benchmark datasets using label names as the only supervision . |
Self-Training with Weak Supervision (2021.naacl-main)
Copied to clipboard
| Challenge: | State-of-the-art deep neural networks require large amounts of labeled training data that is expensive to obtain or not available for many tasks. |
| Approach: | They propose a weak supervision framework that leverages all available data for a given task . they leverage task-specific unlabeled data through self-training with a model that predicts pseudo-labels for instances that may not be covered by weak rules . |
| Outcome: | The proposed framework improves on state-of-the-art datasets on six benchmark tasks. |
Industry Scale Semi-Supervised Learning for Natural Language Understanding (2021.naacl-industry)
Copied to clipboard
| Challenge: | Obtaining human annotation is expensive and time-consuming process. |
| Approach: | They propose a semi-supervised learning pipeline which leverages millions of unlabeled examples to improve natural language understanding tasks. |
| Outcome: | The proposed pipeline can be used to improve natural language understanding tasks. |
LLM-Guided Co-Training for Text Classification (2025.emnlp-main)
Copied to clipboard
| Challenge: | Empirical results show that it achieves state-of-the-art performance on 4 out of 5 benchmark datasets and ranks first among 14 compared methods according to the Friedman test. |
| Approach: | They propose a weighted co-training approach that is guided by Large Language Models (LLMs) they use LLM labels on unlabeled data as target labels and co-train two encoder-only based networks that train each other over multiple iterations. |
| Outcome: | The proposed approach outperforms conventional methods on 4 out of 5 benchmark datasets and ranks first among 14 compared methods according to the Friedman test. |