TransPrompt: Towards an Automatic Transferable Prompting Framework for Few-shot Text Classification (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show that prompts improve performance of large pre-trained language models for few-shot text classification. |
| Approach: | They propose a prompt-based framework for few-shot learning that captures cross-task transferable knowledge and uses two de-biasing techniques to make it more task-agnostic and unbiased . |
| Outcome: | The proposed framework outperforms strong baselines over multiple NLP tasks and datasets. |
Similar Papers
MetaPrompting: Learning to Learn Better Prompts (2022.coling-1)
Copied to clipboard
| Challenge: | Recent research on prompting moves from discrete tokens based "hard prompts" to continuous "soft prompts", which employ learnable vectors as pseudo prompt tokens and achieve better performance. |
| Approach: | They propose a generalized soft prompting method that uses model-agnostic meta-learning to find better initialization for soft prompts. |
| Outcome: | The proposed method improves on three datasets and brings new state-of-the-art performance. |
Prompt-Based Meta-Learning For Few-shot Text Classification (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to learn text labels require large amounts of data to build many few-shot tasks. |
| Approach: | They propose a Prompt-Based Meta-Learning model that adds the prompting mechanism to the meta-learning method. |
| Outcome: | The proposed method improves on four text classification datasets with high accuracy and robustness. |
AdaPrompt: Adaptive Model Training for Prompt-based NLP (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Prompt-based learning can tackle zero-shot and few-shot NLP tasks . authors propose a method that makes use of pre-trained language models . |
| Approach: | They propose to map NLP tasks into natural language prompts, which are then filled by pre-trained language models. |
| Outcome: | The proposed method outperforms standard prompt-based methods in few-shot settings. |
OpenPrompt: An Open-source Framework for Prompt-learning (2022.acl-demo)
Copied to clipboard
| Challenge: | Prompt-learning is a new paradigm in natural language processing, adapting pre-trained language models to cloze-style prediction, autoregressive modeling, or sequence to sequence generation. |
| Approach: | They propose a framework for prompt-learning that integrates pre-trained language models with a unified framework. |
| Outcome: | The proposed framework is easy to use and flexible enough to integrate with other frameworks. |
SwitchPrompt: Learning Domain-Specific Gated Soft Prompts for Classification in Low-Resource Domains (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent work shows promising results when prompting pre-trained language models, but in low-resource domains, the domain gap between the pre-training data and the downstream task is too large. |
| Approach: | They propose a method for prompting pre-trained language models using domain-specific keywords with a trainable gated prompt. |
| Outcome: | The proposed prompting method outperforms state-of-the-art prompting methods on three text classification benchmarks and shows that it reduces the need for domain-specific language model pre-training. |
Enhancing Cross-lingual Prompting with Dual Prompt Augmentation (2023.findings-acl)
Copied to clipboard
| Challenge: | a recent study shows that prompting is superior for multilingual/cross-lingual problems . despite its effectiveness on English tasks, its potential for cross-lingual problem is under-explored . |
| Approach: | They propose a framework for prompting that can be used to augment cross-lingual prompts. |
| Outcome: | The proposed framework achieves 46.54% with only 16 English training examples per class, significantly better than fine-tuning. |
Towards Unified Prompt Tuning for Few-shot Text Classification (2022.findings-emnlp)
Copied to clipboard
Jianing Wang, Chengyu Wang, Fuli Luo, Chuanqi Tan, Minghui Qiu, Fei Yang, Qiuhui Shi, Songfang Huang, Ming Gao
| Challenge: | Prompt-based fine-tuning has boosted performance of Pre-trained Language Models (PLMs) on few-shot text classification, but PLMs are unfamiliar with prompt-style expressions during pre-training, which limits the few- shot learning performance on downstream tasks. |
| Approach: | They propose a framework for prompt-based fine-tuning that captures prompting semantics from non-target NLP datasets and propose 'Prompt-Options-Verbalizer' for joint prompt learning across different NLP tasks. |
| Outcome: | Experiments show that the proposed framework outperforms state-of-the-art prompt-based fine-tuning frameworks on few-shot text classification tasks. |
Self-supervised Meta-Prompt Learning with Meta-Gradient Regularization for Few-shot Generalization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for prompt tuning can overfit to few-shot training samples, causing overfitting . authors propose a new framework for prompt learning with supervised meta-learning . |
| Approach: | They propose a self-supervised meta-prompt learning framework with MEta-gradient Regularization for few-shot generalization that leverages self-recognized meta-learning with a diverse set of meta-tasks to learn a universal prompt initialization using only unlabeled data. |
| Outcome: | The proposed framework learns a universal prompt initialization for efficient adaptation using only unlabeled data. |
Bayesian Multi-Task Transfer Learning for Soft Prompt Tuning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large-scale pre-trained language models have been fine-tuned for various NLP tasks . prompt tuning is a method that optimizes the output of the model to adapt to downstream tasks based on the posterior distribution of the source task. |
| Approach: | They propose a Bayesian approach to prompt tuning that optimizes for adapting pre-trained language models to downstream tasks rather than fine-tuning full model parameters. |
| Outcome: | The proposed approach outperforms the state-of-the-art methods on benchmark NLP tasks. |
FPT: Feature Prompt Tuning for Few-shot Readability Assessment (2024.naacl-long)
Copied to clipboard
| Challenge: | Prompt-based methods lack crucial linguistic knowledge for readability assessment tasks such as word length, sentence length, and usage of different difficulty-level words. |
| Approach: | They propose a new prompt-based tuning framework that incorporates linguistic knowledge and a loss function to calibrate the similarity ranking order between categories. |
| Outcome: | The proposed framework outperforms the large language model gpt-3.5-turbo-16k in most cases. |