Lightweight Contenders: Navigating Semi-Supervised Text Mining through Peer Collaboration and Self Transcendence (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing frameworks for semi-supervised text mining with lightweight models are limited by label data scarcity. |
| Approach: | They propose a framework for semi-supervised text mining with lightweight models . it incorporates online distillation to train lightweight student models by imitating the Teacher model . |
| Outcome: | The proposed framework exhibits notable performance enhancements over existing frameworks. |
Similar Papers
DisCo: Distilled Student Models Co-training for Semi-supervised Text Mining (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing text mining models are fine-tuned by fine-timing a large pre-trained language model (PLM) in downstream tasks. |
| Approach: | They propose a semi-supervised learning framework for fine-tuning a cohort of small student models generated from a large pre-trained language model using knowledge distillation. |
| Outcome: | The proposed framework outperforms baseline models on semi-supervised text classification and extractive summarization tasks while maintaining comparable performance. |
Leveraging Training Dynamics and Self-Training for Text Classification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Semi-supervised learning (SSL) is a promising technique for improving deep learning models when training data is scarce. |
| Approach: | They propose a semi-supervised learning approach that leverages training dynamics of unlabeled data. |
| Outcome: | The proposed method achieves an average increase in F1 score of 3.5% over baselines in low resource settings. |
FLiText: A Faster and Lighter Semi-Supervised Text Classification with Convolution Networks (2021.emnlp-main)
Copied to clipboard
| Challenge: | obtaining large amounts of labeled data is expensive. |
| Approach: | They develop a semi-supervised learning framework called FLiText which improves text classification accuracy. |
| Outcome: | The proposed framework improves accuracy of lightweight models on IMDb, Yelp-5, and Yahoo! Answer . the framework improve accuracy by 6.59%, 3.94%, and 3.22% on the datasets of IMDa, Yep-5 and Yahoo. Answer compared with the fully supervised method on the full dataset . |
Rethinking Semi-supervised Learning with Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Semi-supervised learning (SSL) is a popular setting to make use of unlabelled data . Currently, there are two popular approaches to make effective use of the unlabelled datasets . |
| Approach: | They compare semi-supervised learning (SSL) and task-adaptive pre-training (TAPT) they find TAPT is a stronger and more robust SSL learner, even when using just a few hundred unlabelled samples . |
| Outcome: | The proposed methods improve model performance across different NLP tasks and data sizes. |
Industry Scale Semi-Supervised Learning for Natural Language Understanding (2021.naacl-industry)
Copied to clipboard
| Challenge: | Obtaining human annotation is expensive and time-consuming process. |
| Approach: | They propose a semi-supervised learning pipeline which leverages millions of unlabeled examples to improve natural language understanding tasks. |
| Outcome: | The proposed pipeline can be used to improve natural language understanding tasks. |
Learning to Infer from Unlabeled Data: A Semi-supervised Learning Approach for Robust Natural Language Inference (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Semi-supervised learning (SSL) is a popular technique for reducing the reliance on human annotations for NLI tasks. |
| Approach: | They propose a way to incorporate unlabeled data into semi-supervised learning (SSL) using a conditional language model, they propose to generate hypotheses for unlabed sentences . |
| Outcome: | The proposed framework significantly improves the performance of four NLI datasets in low-resource settings. |
Combining Weakly Supervised ML Techniques for Low-Resource NLU (2021.naacl-industry)
Copied to clipboard
| Challenge: | Recent advances in transfer learning have improved the performance of virtual assistants . however, meager training data is often a key bottleneck in creating voice-enabled applications . |
| Approach: | They propose to use unsupervised and semi-supervised techniques to improve NLU accuracy . they incorporate anonymized, unlabeled and automatically transcribed user utterances into training . |
| Outcome: | The proposed methods improve NLU accuracy in low-resource settings by integrating unsupervised and SSL techniques. |
Combining Deep Generative Models and Multi-lingual Pretraining for Semi-supervised Document Classification (2021.eacl-main)
Copied to clipboard
| Challenge: | Semi-supervised learning and multilingual pretraining have been shown to be effective for task-specific labelled data shortages. |
| Approach: | They propose to combine semi-supervised deep generative models and multi-lingual pretraining to form a pipeline for document classification task. |
| Outcome: | The proposed method outperforms state-of-the-art models in low-resource settings across several languages and outperformed existing models in English. |
Self-Training with Weak Supervision (2021.naacl-main)
Copied to clipboard
| Challenge: | State-of-the-art deep neural networks require large amounts of labeled training data that is expensive to obtain or not available for many tasks. |
| Approach: | They propose a weak supervision framework that leverages all available data for a given task . they leverage task-specific unlabeled data through self-training with a model that predicts pseudo-labels for instances that may not be covered by weak rules . |
| Outcome: | The proposed framework improves on state-of-the-art datasets on six benchmark tasks. |
LLM-Guided Co-Training for Text Classification (2025.emnlp-main)
Copied to clipboard
| Challenge: | Empirical results show that it achieves state-of-the-art performance on 4 out of 5 benchmark datasets and ranks first among 14 compared methods according to the Friedman test. |
| Approach: | They propose a weighted co-training approach that is guided by Large Language Models (LLMs) they use LLM labels on unlabeled data as target labels and co-train two encoder-only based networks that train each other over multiple iterations. |
| Outcome: | The proposed approach outperforms conventional methods on 4 out of 5 benchmark datasets and ranks first among 14 compared methods according to the Friedman test. |