Few-Shot Dialogue Summarization via Skeleton-Assisted Prompt Transfer in Prompt Tuning (2024.eacl-long)
Copied to clipboard
Kaige Xie, Tong Yu, Haoliang Wang, Junda Wu, Handong Zhao, Ruiyi Zhang, Kanak Mahadik, Ani Nenkova, Mark Riedl
| Challenge: | Existing prompt transfer techniques lack consideration for dialogue-specific information. |
| Approach: | They propose a method which leverages skeleton generation as extra supervision that functions as a medium connecting the distinct source and target task. |
| Outcome: | The proposed method significantly outperforms baselines on two dialogue summarization benchmarks. |
Similar Papers
Dialogue Summaries as Dialogue States (DS2), Template-Guided Summarization for Few-shot Dialogue State Tracking (2022.findings-acl)
Copied to clipboard
| Challenge: | Annotating task-oriented dialogues is notorious for the expensive and difficult data collection process. |
| Approach: | They propose to reformulate dialogue state tracking as a dialogue summarization problem by using synthetic dialogue summaries generated by a set of rules. |
| Outcome: | The proposed method outperforms previous studies on few-shot dialogue state tracking in MultiWoZ 2.0 and 2.1 in cross-domain and multi-domain settings. |
Few-shot fine-tuning SOTA summarization models for medical dialogues (2022.naacl-srw)
Copied to clipboard
| Challenge: | Abstractive summarization of medical dialogues is a challenge for standard training approaches due to the paucity of suitable datasets. |
| Approach: | They propose to use medical dialogues to generate abstractive summaries using transformer-based models with zero-shot and few-shot learning strategies. |
| Outcome: | The proposed models were compared with a medical dialogue dataset with 143 snippets and a general domain and dialogue-specific text to assess their performance. |
SynthDST: Synthetic Data is All You Need for Few-Shot Dialog State Tracking (2024.eacl-long)
Copied to clipboard
| Challenge: | In-context learning with Large Language Models (LLMs) is a promising avenue of research in Dialog State Tracking (DST). |
| Approach: | They propose a data generation framework tailored for Dialog State Tracking that uses large language models to synthesize natural, coherent, and free-flowing dialogues with DST annotations. |
| Outcome: | The proposed framework improves joint goal accuracy by 4-5% over the zero-shot baseline on MultiWOZ 2.1 and 2.4. |
PSP: Pre-trained Soft Prompts for Few-Shot Abstractive Summarization (2022.coling-1)
Copied to clipboard
| Challenge: | Experimental results show that our method outperforms full-model tuning in few-shot abstractive summarization tasks. |
| Approach: | They propose a soft prompts architecture with prompt pre-training and prompt fine-tuning paradigm to support few-shot abstractive summarization. |
| Outcome: | The proposed model outperforms Prompt Tuning and Profix-Tuning on CNN/DailyMail and XSum datasets and outperfies Profix Tuning by a large margin. |
Towards Unified Prompt Tuning for Few-shot Text Classification (2022.findings-emnlp)
Copied to clipboard
Jianing Wang, Chengyu Wang, Fuli Luo, Chuanqi Tan, Minghui Qiu, Fei Yang, Qiuhui Shi, Songfang Huang, Ming Gao
| Challenge: | Prompt-based fine-tuning has boosted performance of Pre-trained Language Models (PLMs) on few-shot text classification, but PLMs are unfamiliar with prompt-style expressions during pre-training, which limits the few- shot learning performance on downstream tasks. |
| Approach: | They propose a framework for prompt-based fine-tuning that captures prompting semantics from non-target NLP datasets and propose 'Prompt-Options-Verbalizer' for joint prompt learning across different NLP tasks. |
| Outcome: | Experiments show that the proposed framework outperforms state-of-the-art prompt-based fine-tuning frameworks on few-shot text classification tasks. |
Skeleton-to-Response: Dialogue Generation Guided by Retrieval Memory (N19-1)
Copied to clipboard
| Challenge: | Existing generative dialogue models generate responses from input queries . however, the results are limited and the models are unsatisfactory . |
| Approach: | They propose a framework which exploits retrieval results via a skeleton-to-response paradigm . they extract a query skelet and use it to generate a new skele and response . |
| Outcome: | The proposed approach significantly improves the informativeness of the generated responses. |
The Power of Scale for Parameter-Efficient Prompt Tuning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Unlike discrete text prompts used by GPT-3, soft prompts are learned through backpropagation and can be tuned to incorporate signals from any number of labeled examples. |
| Approach: | They propose a mechanism for learning "soft prompts" to condition frozen language models to perform specific downstream tasks. |
| Outcome: | The proposed method outperforms fewshot learning using GPT-3 and matches the quality of model tuning as models exceed billions of parameters. |
UNO-DST: Leveraging Unlabelled Data in Zero-Shot Dialogue State Tracking (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for zero-shot dialogue state tracking (DST) ignore unlabelled data in the target domain. |
| Approach: | They propose to transform zero-shot dialogue state tracking into few-shot DST by utilising unlabelled data via joint and self-training methods. |
| Outcome: | The proposed method improves joint goal accuracy by 8% on general language models in zero-shot scenarios, and can be used in many domains. |
PPT: Pre-trained Prompt Tuning for Few-shot Learning (2022.acl-long)
Copied to clipboard
| Challenge: | Prompt tuning for pre-trained language models has shown remarkable performance . however, prompt tuning is still not fully explored . |
| Approach: | They propose to pre-train prompts by adding soft prompts into the pre-training stage to obtain a better initialization. |
| Outcome: | The proposed framework outperforms full-model tuning under full-data and few-shot learning settings. |
Prompt Combines Paraphrase: Teaching Pre-trained Models to Understand Rare Biomedical Words (2022.coling-1)
Copied to clipboard
Haochun Wang, Chi Liu, Nuwa Xi, Sendong Zhao, Meizhi Ju, Shiwei Zhang, Ziheng Zhang, Yefeng Zheng, Bing Qin, Ting Liu
| Challenge: | Pre-trained models perform poorly with limited data and rare biomedical words. |
| Approach: | They propose to use prompt to fine-tune pre-trained models for biomedical domain tuning with a simple approach. |
| Outcome: | The proposed method achieves up to 6% improvement in biomedical natural language inference task without any extra parameters or training steps using few-shot vanilla prompt settings. |