Papers with pseudo-labeling
Friend-training: Learning from Models of Different but Related Tasks (2023.eacl-main)
Copied to clipboard
| Challenge: | Current self-training methods focus on improving model performance on a single task. |
| Approach: | They propose a cross-task self-training framework where models trained to do different tasks are used in iterative training, pseudo-labeling, and retraining processes to help each other for better selection of pseudo-labeled labels. |
| Outcome: | The proposed framework achieves the best performance compared to baselines on two dialogue understanding tasks. |
Pseudo2Real: Task Arithmetic for Pseudo-Label Correction in Automatic Speech Recognition (2026.findings-acl)
Copied to clipboard
| Challenge: | ASR models can be used to correct accent-specific errors without ground truth . pseudo-labels inherit the teacher model's systematic biases, authors say . |
| Approach: | They propose a parameter-space correction technique that captures pseudo-label biases . they propose achieving up to 35% relative WER reduction on a pseudo-labeled target model . |
| Outcome: | The proposed model achieves 35% relative WER reduction on ten African accents with the Whisper tiny model. |
Self-Criticism: Aligning Large Language Models with their Understanding of Helpfulness, Honesty, and Harmlessness (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Recent studies have shown that large language models are useful, honest, harmless (HHH) however, RLHF requires high hardware resources and human efforts. |
| Approach: | They propose a framework that allows LLMs to align themselves with HHH . they use IF and reinforcement learning from human feedback to fine-tune their models . |
| Outcome: | The proposed framework achieves similar performance to RLHF and human-generated models with a minimal alignment tax. |
Towards Computationally Feasible Deep Active Learning (2022.findings-naacl)
Copied to clipboard
Akim Tsvigun, Artem Shelmanov, Gleb Kuzmin, Leonid Sanochkin, Daniil Larionov, Gleb Gusev, Manvel Avetisian, Leonid Zhukov
| Challenge: | Active learning (AL) is a technique for reducing the amount of annotation required for training machine learning models. |
| Approach: | They propose two techniques that reduce the amount of time required for AL . they use pseudo-labeling and distilled models to train a successor model . |
| Outcome: | The proposed algorithm reduces the time and computational overhead required to train an acquisition model and estimate uncertainty on instances in the unlabeled pool. |
Weakly-Supervised Methods for Suicide Risk Assessment: Role of Related Domains (2021.acl-short)
Copied to clipboard
| Challenge: | Among social media platforms, Reddit has emerged as the most promising one due to its anonymity and its focus on topic-based communities (subreddits) . a challenge for previous work on suicide risk assessment has been the small amount of labeled data. |
| Approach: | They propose to use social media to collect user data from r/SuicideWatch subreddit and annotate it with user-level suicide risk: no-risk, low-risk and high-risk. |
| Outcome: | The proposed model improves by using pseudo-labeling based on related issues around mental health (e.g., anxiety, depression) |
Unraveling the Dynamics of Semi-Supervised Hate Speech Detection: The Impact of Unlabeled Data Characteristics and Pseudo-Labeling Strategies (2024.findings-eacl)
Copied to clipboard
| Challenge: | Semi-supervised learning addresses the need for large amounts of labeled training data for state-of-the-art approaches. |
| Approach: | They propose to leverage unlabeled data to reduce the amount of annotated data required for machine learning based hate speech detection by using a semi-supervised approach. |
| Outcome: | The proposed approach reduces the amount of annotated data required by state-of-the-art models by leveraging unlabeled data. |
MultiMatch: Multihead Consistency Regularization Matching for Semi-Supervised Text Classification (2025.emnlp-main)
Copied to clipboard
| Challenge: | **MultiMatch** is a semi-supervised learning (SSL) algorithm that combines co-training and consistency regularization with pseudo-labeling. |
| Approach: | They propose a semi-supervised learning algorithm that integrates co-training and consistency regularization with pseudo-labeling. |
| Outcome: | The proposed algorithm outperforms the second-best approach on 8 out of 10 setups from 5 natural language processing datasets and outperformed the second best by 3.26%. |
Domain Adaptation for Question Answering via Question Classification (2022.coling-1)
Copied to clipboard
| Challenge: | Question answering systems often experience performance deterioration upon user-generated questions. |
| Approach: | They propose a question classification framework to help QA domains adapt to different domains. |
| Outcome: | The proposed framework improves on state-of-the-art datasets against multiple datasets. |
Cross-Lingual Summarization with Pseudo-Label Regularization (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to cross-lingual summarization use only a single reference, resulting in an underrepresented hypothesis space. |
| Approach: | They propose to use pseudo-labels to regularize cross-lingual summarization training by combining a single reference and a network to perform the model training. |
| Outcome: | The proposed approach significantly improves over gold reference training in XLS with 8 languages from different families. |
CO3: Low-resource Contrastive Co-training for Generative Conversational Query Rewrite (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent advances in conversational IR systems have seen a resurgent interest in conversation . generative query rewrite generates reconstructed query based on the conversation history . |
| Approach: | They propose to use unlabeled data to make further improvements using contrastive co-training paradigm. |
| Outcome: | The proposed model is robust to noise and language style shift under few-shot and zero-shot scenarios. |
Speech-to-Speech Translation for a Real-world Unwritten Language (2023.findings-acl)
Copied to clipboard
Peng-Jen Chen, Kevin Tran, Yilin Yang, Jingfei Du, Justine Kao, Yu-An Chung, Paden Tomasello, Paul-Ambroise Duquenne, Holger Schwenk, Hongyu Gong, Hirofumi Inaguma, Sravya Popuri, Changhan Wang, Juan Pino, Wei-Ning Hsu, Ann Lee
| Challenge: | a new study examines speech-to-speech translation (S2ST) that translates speech from one language into another . the research area for unwritten languages remains a research area with little exploration due to the lack of training data. |
| Approach: | They propose a system that translates speech from one language into another . they use Taiwanese Hokkien as an example of an unwritten language . |
| Outcome: | The proposed system can be used to train models in languages without standard writing systems. |
Learning Only from Relevant Keywords and Unlabeled Documents (D19-1)
Copied to clipboard
| Challenge: | Existing methods for document classification are limited due to labeling and privacy concerns. |
| Approach: | They propose a super-vised text classification framework that provides keywords as a hint for classifying a document to a target class. |
| Outcome: | The proposed framework is simple to implement and has flexible choices of models, e.g., linear models or neural networks. |
Semi-Supervised Reward Modeling via Iterative Self-Training (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Reward models capture values and preferences of humans and are used in Reinforcement Learning with Human Feedback (RLHF) Traditionally, training large language models relies on extensive human-annotated preference data, which poses significant challenges in terms of scalability and cost. |
| Approach: | They propose a method that enhances RM training using unlabeled data. |
| Outcome: | The proposed approach improves reward models without incurring additional labeling costs on unlabeled datasets. |
JointMatch: A Unified Approach for Diverse and Collaborative Pseudo-Labeling to Semi-Supervised Text Classification (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to semi-supervised text classification suffer from pseudo-label bias and error accumulation. |
| Approach: | They propose a pseudo-labeling approach to semi-supervised text classification that unifies ideas from semi-semi-supervised learning and the task of learning with noise. |
| Outcome: | The proposed approach achieves a significant improvement on benchmark datasets even in the extremely-scarce-label setting. |
Joint Speech Transcription and Translation: Pseudo-Labeling with Out-of-Distribution Data (2023.findings-acl)
Copied to clipboard
| Challenge: | a recent study shows that self-training can improve upon fully supervised baselines in low-resource settings for several sequence-to-sequence tasks. |
| Approach: | They propose to use pseudo-labeling to label unsupervised data and add it to the training pool. |
| Outcome: | The proposed setup improves on the unsupervised data by using pseudo-labeling . the proposed setup provides 0.4% absolute WER and 2.1 BLEU points for En–De . |
Noise-injected Consistency Training and Entropy-constrained Pseudo Labeling for Semi-supervised Extractive Summarization (2022.coling-1)
Copied to clipboard
| Challenge: | Existing studies on semi-supervised learning methods focus on how to effectively utilize abundant unlabeled data. |
| Approach: | They propose a semi-supervised consistency training method to regularize model predictions and a pseudo-labeling strategy to obtain high-confidence labels from unlabeled predictions. |
| Outcome: | The proposed method improves extractive summarization over an insufficient labeled dataset. |
Calibrating Pseudo-Labeling with Class Distribution for Semi-supervised Text Classification (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies develop effective pseudo-labeling methods, but they struggle with unlabeled data that have imbalanced classes mismatched with the labeled data. |
| Approach: | They propose to use pseudo-labeling to train text classification models with few labeled data and massive unlabeled data. |
| Outcome: | Empirical results show that the proposed model outperforms state-of-the-art methods on 3 common benchmarks. |
Entailment as Robust Self-Learner (2023.acl-long)
Copied to clipboard
| Challenge: | Recent studies have found that entailment pretraining benefits weakly supervised fine-tuning. |
| Approach: | They propose a prompting strategy that formulates different NLU tasks as contextual entailment and propose an algorithm for better pseudo-labeling quality in self-training. |
| Outcome: | The proposed approach improves the zero-shot adaptation performance on downstream tasks. |
Understanding and Mitigating Spurious Signal Amplification in Test-Time Reinforcement Learning for Math Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | a framework to mitigate spurious optimization signals is proposed for test-time reinforcement learning (TTRL) Reinforcement learning with verifiable rewards (RLVR) is an effective paradigm for improving large language models on structured challenging reasoning tasks. |
| Approach: | They propose a framework to mitigate spurious optimization signals from label noise . they propose to use a frequency-based sampling strategy to exclude ambiguous samples . |
| Outcome: | The proposed framework outperforms existing TTRL baselines on three large language models across multiple mathematical reasoning benchmarks. |