PSP: Pre-trained Soft Prompts for Few-Shot Abstractive Summarization (2022.coling-1)
Copied to clipboard
| Challenge: | Experimental results show that our method outperforms full-model tuning in few-shot abstractive summarization tasks. |
| Approach: | They propose a soft prompts architecture with prompt pre-training and prompt fine-tuning paradigm to support few-shot abstractive summarization. |
| Outcome: | The proposed model outperforms Prompt Tuning and Profix-Tuning on CNN/DailyMail and XSum datasets and outperfies Profix Tuning by a large margin. |
Similar Papers
PPT: Pre-trained Prompt Tuning for Few-shot Learning (2022.acl-long)
Copied to clipboard
| Challenge: | Prompt tuning for pre-trained language models has shown remarkable performance . however, prompt tuning is still not fully explored . |
| Approach: | They propose to pre-train prompts by adding soft prompts into the pre-training stage to obtain a better initialization. |
| Outcome: | The proposed framework outperforms full-model tuning under full-data and few-shot learning settings. |
Making Pre-trained Language Models Better Few-shot Learners (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that the GPT-3 model can perform few-shots on language understanding tasks with a natural-language prompt and a few task demonstrations. |
| Approach: | They propose a technique for fine-tuning language models using a few examples . they propose LM-BFF, which uses prompt-based fine-uning and a pipeline for automating prompt generation . |
| Outcome: | The proposed approach outperforms standard fine-tuning procedures on a range of NLP tasks. |
StablePT : Towards Stable Prompting for Few-shot Learning via Input Separation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on prompt tuning have shown that language models can be effective few-shot learners with prompting. |
| Approach: | They propose to treat the hard prompt and soft prompt as separate inputs to mitigate noise brought by prompt initialization. |
| Outcome: | Experimental results show that the proposed method outperforms state-of-the-art methods by 6.97% in accuracy and reduces the standard deviation by 1.92 on average. |
Towards Unified Prompt Tuning for Few-shot Text Classification (2022.findings-emnlp)
Copied to clipboard
Jianing Wang, Chengyu Wang, Fuli Luo, Chuanqi Tan, Minghui Qiu, Fei Yang, Qiuhui Shi, Songfang Huang, Ming Gao
| Challenge: | Prompt-based fine-tuning has boosted performance of Pre-trained Language Models (PLMs) on few-shot text classification, but PLMs are unfamiliar with prompt-style expressions during pre-training, which limits the few- shot learning performance on downstream tasks. |
| Approach: | They propose a framework for prompt-based fine-tuning that captures prompting semantics from non-target NLP datasets and propose 'Prompt-Options-Verbalizer' for joint prompt learning across different NLP tasks. |
| Outcome: | Experiments show that the proposed framework outperforms state-of-the-art prompt-based fine-tuning frameworks on few-shot text classification tasks. |
Zero-shot Approach to Overcome Perturbation Sensitivity of Prompts (2023.acl-long)
Copied to clipboard
| Challenge: | Recent studies have demonstrated that natural-language prompts can help to leverage the knowledge learned by pre-trained language models for the binary sentence-level sentiment classification task. |
| Approach: | They propose to use few-shot learning settings to fine-tune the sentiment classification model using manual or automatically generated prompts. |
| Outcome: | The proposed method outperforms the base prompt and the prompts generated using few-shot learning for the binary sentence-level sentiment classification task. |
PELMS: Pre-training for Effective Low-Shot Multi-Document Summarization (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for abstractive multi-document summarization fail to generate concise, reflective summaries. |
| Approach: | They propose a pre-trained abstractive multi-document summarization model that uses unlabeled multi-doctoral inputs to generate concise, reflective summaries. |
| Outcome: | The proposed model outperforms competing models on a wide range of MDS datasets. |
Improving Zero and Few-Shot Abstractive Summarization with Intermediate Fine-tuning and Data Augmentation (2021.naacl-main)
Copied to clipboard
Alexander Fabbri, Simeng Han, Haoyuan Li, Haoran Li, Marjan Ghazvininejad, Shafiq Joty, Dragomir Radev, Yashar Mehdad
| Challenge: | Recent advances in summarization are driven by the availability of large datasets such as the CNN-DailyMail corpus and the New York Times corpus. |
| Approach: | They propose a method for fine-tuning pretrained models for summarization in unsupervised manner . they use Wikipedia data to produce pseudo-summaries which contain characteristics of target dataset . |
| Outcome: | The proposed method achieves state-of-the-art, zero-shot abstractive summarization performance on CNN-DailyMail dataset and compares with other methods on other datasets. |
Multitask Pre-training of Modular Prompt for Chinese Few-Shot Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Prompt tuning is a parameter-efficient approach to adapting pre-trained language models to downstream tasks. |
| Approach: | They propose to combine pre-trained modules with pre-trains to boost prompt tuning for few-shot learning. |
| Outcome: | The proposed model outperforms prompt tuning, full model tuning, and prior prompt pre-training methods in few-shot learning settings. |
Efficient Few-Shot Fine-Tuning for Opinion Summarization (2022.findings-naacl)
Copied to clipboard
| Challenge: | Abstractive summarization models are typically pre-trained on large amounts of generic texts . large annotated datasets of reviews paired with reference summaries are not available . |
| Approach: | They propose a few-shot method which uses adapters to store in-domain knowledge . they pre-train adapters on unannotated customer reviews and fine-tune them on annotated datasets . |
| Outcome: | The proposed method can store in-domain knowledge and improves on large annotated reviews . it improves coherence and redundancies on the Amazon and Yelp datasets . |
Visual Prompt Tuning for Few-Shot Text Classification (2022.coling-1)
Copied to clipboard
| Challenge: | Existing work on pretraining models for text classification uses image encoders instead of visual prompts. |
| Approach: | They propose a method to deploy large-scale pre-trained models in the prompt-tuning paradigm in few-shot learning. |
| Outcome: | The proposed method outperforms the most recent prompt-tuning methods on five public text classification datasets. |