FPT: Feature Prompt Tuning for Few-shot Readability Assessment (2024.naacl-long)
Copied to clipboard
| Challenge: | Prompt-based methods lack crucial linguistic knowledge for readability assessment tasks such as word length, sentence length, and usage of different difficulty-level words. |
| Approach: | They propose a new prompt-based tuning framework that incorporates linguistic knowledge and a loss function to calibrate the similarity ranking order between categories. |
| Outcome: | The proposed framework outperforms the large language model gpt-3.5-turbo-16k in most cases. |
Similar Papers
PPT: Pre-trained Prompt Tuning for Few-shot Learning (2022.acl-long)
Copied to clipboard
| Challenge: | Prompt tuning for pre-trained language models has shown remarkable performance . however, prompt tuning is still not fully explored . |
| Approach: | They propose to pre-train prompts by adding soft prompts into the pre-training stage to obtain a better initialization. |
| Outcome: | The proposed framework outperforms full-model tuning under full-data and few-shot learning settings. |
Towards Unified Prompt Tuning for Few-shot Text Classification (2022.findings-emnlp)
Copied to clipboard
Jianing Wang, Chengyu Wang, Fuli Luo, Chuanqi Tan, Minghui Qiu, Fei Yang, Qiuhui Shi, Songfang Huang, Ming Gao
| Challenge: | Prompt-based fine-tuning has boosted performance of Pre-trained Language Models (PLMs) on few-shot text classification, but PLMs are unfamiliar with prompt-style expressions during pre-training, which limits the few- shot learning performance on downstream tasks. |
| Approach: | They propose a framework for prompt-based fine-tuning that captures prompting semantics from non-target NLP datasets and propose 'Prompt-Options-Verbalizer' for joint prompt learning across different NLP tasks. |
| Outcome: | Experiments show that the proposed framework outperforms state-of-the-art prompt-based fine-tuning frameworks on few-shot text classification tasks. |
The Power of Scale for Parameter-Efficient Prompt Tuning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Unlike discrete text prompts used by GPT-3, soft prompts are learned through backpropagation and can be tuned to incorporate signals from any number of labeled examples. |
| Approach: | They propose a mechanism for learning "soft prompts" to condition frozen language models to perform specific downstream tasks. |
| Outcome: | The proposed method outperforms fewshot learning using GPT-3 and matches the quality of model tuning as models exceed billions of parameters. |
Exploiting Language Model Prompts Using Similarity Measures: A Case Study on the Word-in-Context Task (2022.acl-short)
Copied to clipboard
| Challenge: | Existing few-shot approaches fail on the semantic distinction task of the Word-in-Context dataset. |
| Approach: | They propose a prompt-based approach which boosts few-shot performance to the level of fully supervised methods by using similarity metrics. |
| Outcome: | The proposed technique boosts few-shot performance to the level of fully supervised methods. |
Making Pre-trained Language Models Better Few-shot Learners (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that the GPT-3 model can perform few-shots on language understanding tasks with a natural-language prompt and a few task demonstrations. |
| Approach: | They propose a technique for fine-tuning language models using a few examples . they propose LM-BFF, which uses prompt-based fine-uning and a pipeline for automating prompt generation . |
| Outcome: | The proposed approach outperforms standard fine-tuning procedures on a range of NLP tasks. |
StablePT : Towards Stable Prompting for Few-shot Learning via Input Separation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on prompt tuning have shown that language models can be effective few-shot learners with prompting. |
| Approach: | They propose to treat the hard prompt and soft prompt as separate inputs to mitigate noise brought by prompt initialization. |
| Outcome: | Experimental results show that the proposed method outperforms state-of-the-art methods by 6.97% in accuracy and reduces the standard deviation by 1.92 on average. |
Toward Human Readable Prompt Tuning: Kubrick’s The Shining is a good movie, and a good prompt too? (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models can perform downstream tasks in a zero-shot fashion, given natural language prompts that specify the desired behavior. |
| Approach: | They propose a human readable prompt tuning method that incorporates a fluency constraint to find a distribution of effective and fluent prompts. |
| Outcome: | The proposed method outperforms baselines by 7.0% across three tasks. |
FPT: Improving Prompt Tuning Efficiency via Progressive Training (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent prompt tuning (PT) has gained increasing attention as a parameter-efficient way of tuning pre-trained language models (PLMs). |
| Approach: | They propose a prompt tuning algorithm that uses a small-scale partial PLM and progressively expands its depth and width until the full-model size. |
| Outcome: | The proposed method could save over 30% of training computations while achieving comparable performance. |
Visual Prompt Tuning for Few-Shot Text Classification (2022.coling-1)
Copied to clipboard
| Challenge: | Existing work on pretraining models for text classification uses image encoders instead of visual prompts. |
| Approach: | They propose a method to deploy large-scale pre-trained models in the prompt-tuning paradigm in few-shot learning. |
| Outcome: | The proposed method outperforms the most recent prompt-tuning methods on five public text classification datasets. |
Position Really Matters: Towards a Holistic Approach for Prompt Tuning (2025.findings-naacl)
Copied to clipboard
| Challenge: | Prompt tuning is effective in extracting knowledge from foundation models, but its effectiveness is uncertain. |
| Approach: | They propose a parametric prompt tuning strategy that dynamically determines different factors of prompts based on specific tasks or instances. |
| Outcome: | The proposed approach improves performance across a wide range of tasks including NLP, vision recognition, and vision-language tasks. |