Zhiquan Ye, Yuxia Geng, Jiaoyan Chen, Jingmin Chen, Xiaoxiao Xu, SuHang Zheng, Feng Wang, Jun Zhang, Huajun Chen
| Challenge: | Existing methods to learn from unlabeled data are difficult for zero-shot text classification tasks. |
| Approach: | They propose a self-training based method to efficiently leverage unlabeled data. |
| Outcome: | The proposed method significantly outperforms existing methods in zero-shot text classification tasks on benchmarks and a real-world e-commerce dataset. |
Similar Papers
Zero-Shot Text Classification with Self-Training (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large pretrained language models have increased attention to zero-shot text classification. |
| Approach: | They propose a plug-and-play method to bridge this gap by requiring only class names along with an unlabeled dataset. |
| Outcome: | The proposed model can be trained on a natural language inference dataset and performs on dozens of unseen tasks without the need for domain expertise or trial and error. |
Zero-Shot Text Classification via Self-Supervised Tuning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing solutions to zero-shot text classification use pre-trained language models or large-scale annotated data. |
| Approach: | They propose a self-supervised learning paradigm to solve zero-shot text classification tasks by tuning the language models with unlabeled data. |
| Outcome: | The proposed model outperforms the state-of-the-art models on 7 out of 10 tasks and is less sensitive to prompt design. |
A weakly supervised textual entailment approach to zero-shot text classification (2023.eacl-main)
Copied to clipboard
Marc Pàmies, Joan Llop, Francesco Multari, Nicolau Duran-Silva, César Parra-Rojas, Aitor Gonzalez-Agirre, Francesco Alessandro Massucci, Marta Villegas
| Challenge: | Existing methods to train on weakly supervised datasets are expensive due to the computational cost of pre-training. |
| Approach: | They propose a method that trains on a weakly supervised dataset that is used as a proxy for a textual entailment problem and a target zero-shot text classification task. |
| Outcome: | The proposed model achieves state-of-the-art performance in the scientific domain and competitive results in other areas. |
Text2Model: Text-based Model Induction for Zero-shot Image Classification (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to zero-shot learning are limited in two ways: Query-dependence and richness of language description. |
| Approach: | They propose a task-agnostic approach to image classification using only text descriptions . they train a hypernetwork that receives class descriptions and outputs a multi-class model . |
| Outcome: | The proposed approach generates non-linear classifiers, handles rich textual descriptions, and may be adapted to produce lightweight models efficient enough for on-device applications. |
ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrieval (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies show that large pretrained language models can generate training data with no task-specific or cross-task data. |
| Approach: | They propose a retrieval-enhanced framework to create training data from a general-domain unlabeled corpus. |
| Outcome: | The proposed framework achieves 4.3% gain over baselines and saves 70% of time compared with baselines using large language models. |
Leveraging Training Dynamics and Self-Training for Text Classification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Semi-supervised learning (SSL) is a promising technique for improving deep learning models when training data is scarce. |
| Approach: | They propose a semi-supervised learning approach that leverages training dynamics of unlabeled data. |
| Outcome: | The proposed method achieves an average increase in F1 score of 3.5% over baselines in low resource settings. |
Integrating Semantic Knowledge to Tackle Zero-shot Text Classification (N19-1)
Copied to clipboard
| Challenge: | Existing approaches to classify text documents of emerging classes are ineffective because of insufficient or even unavailable training data. |
| Approach: | They propose a two-phase framework with data augmentation and feature augmentation to deal with unseen classes effectively using four kinds of semantic knowledge. |
| Outcome: | The proposed framework achieves the best overall accuracy compared with baselines and recent approaches in classifying real-world texts under the zero-shot scenario. |
PESCO: Prompt-enhanced Self Contrastive Learning for Zero-shot Text Classification (2023.acl-long)
Copied to clipboard
| Challenge: | Existing text classification frameworks require large amounts of human-labeled documents to train . |
| Approach: | They propose a contrastive learning framework that improves zero-shot text classification . they add prompts to enhance label retrieval and use retrieved labels to enrich training . |
| Outcome: | The proposed framework achieves state-of-the-art on four benchmark text classification datasets. |
Label Agnostic Pre-training for Zero-shot Text Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to text classification assume a fixed set of labels . however, in real-world applications, there exists an infinite label space for describing a given text . |
| Approach: | They propose two new methods that inject aspect-level understanding into pre-trained models at train time to improve zero-shot generalization. |
| Outcome: | The proposed methods improve zero-shot generalization on a set of challenging datasets. |
Self-Improving for Zero-Shot Named Entity Recognition with Large Language Models (2024.naacl-short)
Copied to clipboard
| Challenge: | Existing studies exploring the performance of large language models on named entity recognition tasks have focused on training task-specific LLMs for NER. |
| Approach: | They propose a training-free self-improving framework that utilizes an unlabeled corpus to stimulate the self-learning ability of LLMs. |
| Outcome: | The proposed framework improves performance on the named entity recognition task by using an unlabeled corpus. |