PromptDA: Label-guided Data Augmentation for Prompt-based Few Shot Learners (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing studies on prompt-based few-shot tuning focus on deriving proper label words with a verbalizer or generating prompt templates to elicit semantics from PLMs. |
| Approach: | They propose a framework that leverages label semantics for prompt-based tuning. |
| Outcome: | The proposed framework improves on few-shot text classification tasks by leveraging label semantics and data augmentation. |
Similar Papers
LM-CPPF: Paraphrasing-Guided Data Augmentation for Contrastive Prompt-Based Few-Shot Fine-Tuning (2023.acl-short)
Copied to clipboard
| Challenge: | Recent advances in pre-trained language models have been limited when fine-tuned on small datasets. |
| Approach: | They propose to add contrastive learning to prompt-based fine-tuning to improve model performance. |
| Outcome: | The proposed approach outperforms other methods on multiple text classification benchmarks. |
Prompt-Based Meta-Learning For Few-shot Text Classification (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to learn text labels require large amounts of data to build many few-shot tasks. |
| Approach: | They propose a Prompt-Based Meta-Learning model that adds the prompting mechanism to the meta-learning method. |
| Outcome: | The proposed method improves on four text classification datasets with high accuracy and robustness. |
Towards Unified Prompt Tuning for Few-shot Text Classification (2022.findings-emnlp)
Copied to clipboard
Jianing Wang, Chengyu Wang, Fuli Luo, Chuanqi Tan, Minghui Qiu, Fei Yang, Qiuhui Shi, Songfang Huang, Ming Gao
| Challenge: | Prompt-based fine-tuning has boosted performance of Pre-trained Language Models (PLMs) on few-shot text classification, but PLMs are unfamiliar with prompt-style expressions during pre-training, which limits the few- shot learning performance on downstream tasks. |
| Approach: | They propose a framework for prompt-based fine-tuning that captures prompting semantics from non-target NLP datasets and propose 'Prompt-Options-Verbalizer' for joint prompt learning across different NLP tasks. |
| Outcome: | Experiments show that the proposed framework outperforms state-of-the-art prompt-based fine-tuning frameworks on few-shot text classification tasks. |
PromDA: Prompt-based Data Augmentation for Low-Resource NLU Tasks (2022.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to build labeled training data from domain-specific data are expensive to obtain. |
| Approach: | They propose a Prompt-based Data Augmentation model which only trains small-scale Soft Promptes in frozen Pre-trained Language Models. |
| Outcome: | The proposed model outperforms several baseline models on four benchmarks and is complementary with unlabeled in-domain data. |
Revisiting Self-training for Few-shot Learning of Language Model (2021.emnlp-main)
Copied to clipboard
| Challenge: | Unlabeled data are useful for few-shot learning of language models. |
| Approach: | They propose a prompt-based few-shot learner that uses unlabeled data to fine-tune language models. |
| Outcome: | The proposed approach outperforms state-of-the-art models on six sentence classification and six sentence-pair classification benchmarking tasks. |
Prototypical Verbalizer for Prompt-based Few-shot Tuning (2022.acl-long)
Copied to clipboard
| Challenge: | Prompt-based tuning for pre-trained language models has shown its effectiveness in few-shot learning. |
| Approach: | They propose a prototypical verbalizer which learns prototype vectors as verbalizes by contrastive learning. |
| Outcome: | The proposed verbalizer outperforms existing verbalizing methods on topic classification and entity typing tasks. |
Prompt-free and Efficient Few-shot Learning with Language Models (2022.acl-long)
Copied to clipboard
Rabeeh Karimi Mahabadi, Luke Zettlemoyer, James Henderson, Lambert Mathias, Marzieh Saeidi, Veselin Stoyanov, Majid Yazdani
| Challenge: | Existing methods for few-shot fine-tuning of pretrained language models require carefully engineered prompts and verbalizers to convert inputs into a cloze-format that the PLM can score. |
| Approach: | They propose a method for few-shot fine-tuning of pretrained language models that uses task-specific adapters instead of manually engineered prompts and verbalizers. |
| Outcome: | The proposed method outperforms existing state-of-the-art methods on a wide range of few shot NLP tasks. |
Pre-trained Language Models Can be Fully Zero-Shot Learners (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to pre-trained language models require fine-tuning on labeled datasets or manually constructing proper prompts. |
| Approach: | They propose a nonparametric prompting PLM for fully zero-shot language understanding . they compare it to previous methods for text classification and text entailment . |
| Outcome: | The proposed method outperforms previous methods on diverse tasks. |
Adversarial Knowledge Stimulated Contrastive Prompting for Few-shot Language Learners (2023.findings-acl)
Copied to clipboard
| Challenge: | Prompt-based fine-tuning has boosted performance of Pre-trained language models on few-shot Natural Language Understanding (NLU) tasks by employing task-specific prompts. |
| Approach: | They propose a Cloze-driven prompt framework for prompt tuning that implicitly stimulates knowledge from pre-trained language models. |
| Outcome: | The proposed framework outperforms state-of-the-art for prompt-based fine-tuning on few-shot NLU tasks. |
RGL: A Simple yet Effective Relation Graph Augmented Prompt-based Tuning Approach for Few-Shot Learning (2022.findings-naacl)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) are a good starting point for downstream applications, but it is difficult to generalize them to new tasks given a few labeled samples. |
| Approach: | They propose to use Relation Graph augmented learning to improve the performance of few-shot natural language understanding tasks by rewriting the input sequence into a cloze question with masks. |
| Outcome: | Extensive experiments show that Relation Graph augmented learning (RGL) improves performance of prompt-based tuning strategies. |