ZeroAE: Pre-trained Language Model based Autoencoder for Transductive Zero-shot Text Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for text classification use only encoders or decoders that do not allow for the use of labels in unseen domains. |
| Approach: | They propose an autoencoder that encodes text into two disentangled spaces and decodes it to generate text with labels in the unseen domains. |
| Outcome: | The proposed model outperforms the existing methods in label-partially-unseen and label-fully-un-seeen scenarios and even outperfects the SOTA methods. |
Similar Papers
Zero-Shot Text Classification with Self-Training (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large pretrained language models have increased attention to zero-shot text classification. |
| Approach: | They propose a plug-and-play method to bridge this gap by requiring only class names along with an unlabeled dataset. |
| Outcome: | The proposed model can be trained on a natural language inference dataset and performs on dozens of unseen tasks without the need for domain expertise or trial and error. |
Label Agnostic Pre-training for Zero-shot Text Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to text classification assume a fixed set of labels . however, in real-world applications, there exists an infinite label space for describing a given text . |
| Approach: | They propose two new methods that inject aspect-level understanding into pre-trained models at train time to improve zero-shot generalization. |
| Outcome: | The proposed methods improve zero-shot generalization on a set of challenging datasets. |
Don’t Prompt, Search! Mining-based Zero-Shot Learning with Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has obtained strong zero-shot results by prompting language models. |
| Approach: | They propose a mining-based approach that uses regular expressions to mine labeled examples from unlabeled corpora and fine tune a pretrained model. |
| Outcome: | The proposed method outperforms prompting on a wide range of tasks when using comparable templates. |
The Benefits of Label-Description Training for Zero-Shot Text Classification (2023.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained language models have improved zero-shot text classification by allowing the transfer of semantic knowledge from the training data to classify among specific label sets in downstream tasks. |
| Approach: | They propose to use a small finetuning dataset to describe the labels for a task and to use it to further improve zero-shot accuracies. |
| Outcome: | The proposed model is more accurate than zero-shot by 17-19% absolute across topic and sentiment datasets and more robust to choices required for zero- shot classification. |
ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrieval (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies show that large pretrained language models can generate training data with no task-specific or cross-task data. |
| Approach: | They propose a retrieval-enhanced framework to create training data from a general-domain unlabeled corpus. |
| Outcome: | The proposed framework achieves 4.3% gain over baselines and saves 70% of time compared with baselines using large language models. |
Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classification (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to few-shot text classification require domain expertise and an understanding of the language model's abilities to define the mapping between words and labels. |
| Approach: | They propose a method that converts textual inputs to cloze questions that contain some form of task description and processes them with a pretrained language model to map the predicted words to labels. |
| Outcome: | The proposed approach performs almost as well as hand-crafted label-to-word mappings for a number of tasks with small amounts of training data. |
Pre-trained Language Models Can be Fully Zero-Shot Learners (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to pre-trained language models require fine-tuning on labeled datasets or manually constructing proper prompts. |
| Approach: | They propose a nonparametric prompting PLM for fully zero-shot language understanding . they compare it to previous methods for text classification and text entailment . |
| Outcome: | The proposed method outperforms previous methods on diverse tasks. |
ZeroGen: Efficient Zero-shot Learning via Dataset Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to generate training data with pre-trained language models have been found effective in various scenarios. |
| Approach: | They propose an unsupervised zero-shot learning method that generates a dataset from scratch and trains a tiny task model under supervision of the synthesized dataset. |
| Outcome: | The proposed method is annotated-free and efficient, but can provide useful insights from the perspective of data-free model-agnostic knowledge distillation and unreferenced text generation evaluation. |
Zero- and Few-Shot NLP with Pretrained Language Models (2022.acl-tutorials)
Copied to clipboard
| Challenge: | a tutorial aims to introduce NLP researchers to the latest techniques for learning from little-to-no data . aims at bringing interested researchers up to speed about the latest and ongoing techniques . |
| Approach: | They aim to introduce techniques for learning from little-to-no data using pretrained language models. |
| Outcome: | This tutorial aims to bring interested NLP researchers up to speed about recent techniques . it will cover methods from manual engineering, better inference algorithms to better tuning methods . |
Zero-shot Dependency Parsing with Pre-trained Multilingual Sentence Representations (D19-61)
Copied to clipboard
| Challenge: | Pretrained sentence representations have set the new state of the art in many language understanding tasks. |
| Approach: | They propose to use a multilingual corpus to train deep bidirectional sentence representations that are fully lexicalized to allow for the development of an unsupervised universal dependency parser. |
| Outcome: | The proposed approach outperforms the best CoNLL 2018 systems in all of the shared task’s six truly low-resource languages while using a single system. |