Correcting Language Model Bias for Text Classification in True Zero-Shot Learning (2024.lrec-main)
Copied to clipboard
| Challenge: | Experimental results show that pre-trained language models outperform standard prompt learning in zero-shot settings. |
| Approach: | They propose a pipeline for annotating and filtering examples from unlabeled examples . they propose 'model bias validation' method that utilizes unlabed examples as validation set . |
| Outcome: | The proposed approach outperforms standard prompt learning on six text classification tasks. |
Similar Papers
Zero-Shot Text Classification with Self-Training (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large pretrained language models have increased attention to zero-shot text classification. |
| Approach: | They propose a plug-and-play method to bridge this gap by requiring only class names along with an unlabeled dataset. |
| Outcome: | The proposed model can be trained on a natural language inference dataset and performs on dozens of unseen tasks without the need for domain expertise or trial and error. |
Beyond prompting: Making Pre-trained Language Models Better Zero-shot Learners by Clustering Representations (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for zero-shot text classification involve heavy human engineering or complicated self-training pipelines. |
| Approach: | They propose to fit unlabeled text with a Bayesian Gaussian Mixture Model and use class names to cluster them. |
| Outcome: | The proposed approach outperforms prompt-based methods on topic and sentiment datasets and outperformed previous studies significantly on unbalanced datasets. |
Pre-trained Language Models Can be Fully Zero-Shot Learners (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to pre-trained language models require fine-tuning on labeled datasets or manually constructing proper prompts. |
| Approach: | They propose a nonparametric prompting PLM for fully zero-shot language understanding . they compare it to previous methods for text classification and text entailment . |
| Outcome: | The proposed method outperforms previous methods on diverse tasks. |
The Benefits of Label-Description Training for Zero-Shot Text Classification (2023.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained language models have improved zero-shot text classification by allowing the transfer of semantic knowledge from the training data to classify among specific label sets in downstream tasks. |
| Approach: | They propose to use a small finetuning dataset to describe the labels for a task and to use it to further improve zero-shot accuracies. |
| Outcome: | The proposed model is more accurate than zero-shot by 17-19% absolute across topic and sentiment datasets and more robust to choices required for zero- shot classification. |
Label Agnostic Pre-training for Zero-shot Text Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to text classification assume a fixed set of labels . however, in real-world applications, there exists an infinite label space for describing a given text . |
| Approach: | They propose two new methods that inject aspect-level understanding into pre-trained models at train time to improve zero-shot generalization. |
| Outcome: | The proposed methods improve zero-shot generalization on a set of challenging datasets. |
Language Models for Text Classification: Is In-Context Learning Enough? (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing research on text classification models with prompts is limited in scale and lacks understanding of how these methods compare to more established methods. |
| Approach: | They compare the performance of large and smaller language models with prompts to achieve state-of-the-art performance in many NLP tasks. |
| Outcome: | The proposed models outperform the more standard approaches in binary, multiclass, and multilabel tasks in a large scale evaluation of 16 text classification datasets. |
Zero-Shot Text Classification via Self-Supervised Tuning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing solutions to zero-shot text classification use pre-trained language models or large-scale annotated data. |
| Approach: | They propose a self-supervised learning paradigm to solve zero-shot text classification tasks by tuning the language models with unlabeled data. |
| Outcome: | The proposed model outperforms the state-of-the-art models on 7 out of 10 tasks and is less sensitive to prompt design. |
Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Large pre-trained language models (LMs) have a surprising ability to perform zero-shot learning. |
| Approach: | They propose to fine-tune pre-trained language models to optimize the zero-shot learning objective by aggregating 43 existing datasets and annotating 441 label descriptions in a question-answering format. |
| Outcome: | The proposed model outperforms a same-sized QA model and the previous SOTA zero-shot learning system on unseen tasks. |
Issues with Entailment-based Zero-shot Text Classification (2021.acl-short)
Copied to clipboard
| Challenge: | Pre-trained BERT models with no fine-tuning can yield competitive performance against BERT fine- tuned for NLI. |
| Approach: | They propose to use any target label into a sentence of hypothesis and verify whether it could be entailed by the input. |
| Outcome: | The proposed models perform better than models fine-tuned for BERT, but the results are in general negative. |
Zero- and Few-Shot NLP with Pretrained Language Models (2022.acl-tutorials)
Copied to clipboard
| Challenge: | a tutorial aims to introduce NLP researchers to the latest techniques for learning from little-to-no data . aims at bringing interested researchers up to speed about the latest and ongoing techniques . |
| Approach: | They aim to introduce techniques for learning from little-to-no data using pretrained language models. |
| Outcome: | This tutorial aims to bring interested NLP researchers up to speed about recent techniques . it will cover methods from manual engineering, better inference algorithms to better tuning methods . |