| Challenge: | Query-focused summarization has been considered as an important extension for text summarizing . lack of large-scale datasets hinders its development . |
| Approach: | They propose to integrate text summarization and question answering into a prefix-based pretraining strategy for few-shot learning in query-focused summarizing. |
| Outcome: | The proposed prefix-based pretraining outperforms fine-tuning on query-focused summarization. |
Similar Papers
PSP: Pre-trained Soft Prompts for Few-Shot Abstractive Summarization (2022.coling-1)
Copied to clipboard
| Challenge: | Experimental results show that our method outperforms full-model tuning in few-shot abstractive summarization tasks. |
| Approach: | They propose a soft prompts architecture with prompt pre-training and prompt fine-tuning paradigm to support few-shot abstractive summarization. |
| Outcome: | The proposed model outperforms Prompt Tuning and Profix-Tuning on CNN/DailyMail and XSum datasets and outperfies Profix Tuning by a large margin. |
Few-shot fine-tuning SOTA summarization models for medical dialogues (2022.naacl-srw)
Copied to clipboard
| Challenge: | Abstractive summarization of medical dialogues is a challenge for standard training approaches due to the paucity of suitable datasets. |
| Approach: | They propose to use medical dialogues to generate abstractive summaries using transformer-based models with zero-shot and few-shot learning strategies. |
| Outcome: | The proposed models were compared with a medical dialogue dataset with 143 snippets and a general domain and dialogue-specific text to assess their performance. |
FewshotQA: A simple framework for few-shot learning of question answering tasks using pre-trained text-to-text models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-trained models need fine-tuning on tens of thousands of examples to achieve good results. |
| Approach: | They propose a framework that leverages pre-trained text-to-text models and aligns them with their pre-training framework. |
| Outcome: | The proposed framework outperforms the XLM-Roberta-large on multiple QA benchmarks and is applicable to multilingual situations. |
Domain-Oriented Prefix-Tuning: Towards Efficient and Generalizable Fine-tuning for Zero-Shot Dialogue Summarization (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for domain adaptation of abstractive dialogue summarization lack generalization ability on new domains. |
| Approach: | They propose a domain-oriented prefix-tuning model that uses a prefix module to alleviate domain entanglement and discrete prompts to guide the model to focus on key contents of dialogues. |
| Outcome: | The proposed model can be generalized to two multi-domain dialogue summarization datasets. |
Few-shot Table-to-text Generation with Prefix-Controlled Generator (2022.coling-1)
Copied to clipboard
| Challenge: | Neural table-to-text generation approaches are data-hungry and lack labeled data. |
| Approach: | They propose a prompt-based approach for few-shot table-to-text generation using a task-specific prefix and an input-specific input prefix. |
| Outcome: | The proposed approach is able to generate table-to-text summaries with a few instances and is validated on human, book and song datasets. |
Few-Shot Semantic Parsing for New Predicates (2021.eacl-main)
Copied to clipboard
| Challenge: | a recent study shows that state-of-the-art neural semantic parsers are less accurate when there is only a handful of utterance-logical form pairs per predicate. |
| Approach: | They propose to use a meta-learning method to train a few-shot learning problem . they also propose to regularize attention scores with alignment statistics and apply a smoothing technique . |
| Outcome: | The proposed method outperforms baselines in one and two-shot settings. |
Few-shot Transfer Learning for Knowledge Base Question Answering: Fusing Supervised Models with In-Context Learning (2024.acl-long)
Copied to clipboard
| Challenge: | Existing Knowledge Base Question Answering (KBQA) architectures are expensive and time-consuming to deploy. |
| Approach: | They propose a KBQA architecture that performs KB-retrieval using multiple source-trained retrievers and re-ranks using an LLM. |
| Outcome: | The proposed architecture outperforms adaptations of SoTA KBQA models when training data is limited. |
Using dependency parsing for few-shot learning in distributional semantics (2022.acl-srw)
Copied to clipboard
| Challenge: | Existing methods for few-shot learning use dependency parsing information to learn meaning of rare words based on limited amount of context sentences. |
| Approach: | They propose dependency parsing for few-shot learning to learn meaning of rare words . they use word embedding models as background spaces for few shot learning . |
| Outcome: | The proposed methods enhance the additive baseline model by using dependencies. |
Multitask Pre-training of Modular Prompt for Chinese Few-Shot Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Prompt tuning is a parameter-efficient approach to adapting pre-trained language models to downstream tasks. |
| Approach: | They propose to combine pre-trained modules with pre-trains to boost prompt tuning for few-shot learning. |
| Outcome: | The proposed model outperforms prompt tuning, full model tuning, and prior prompt pre-training methods in few-shot learning settings. |
Efficient Few-Shot Fine-Tuning for Opinion Summarization (2022.findings-naacl)
Copied to clipboard
| Challenge: | Abstractive summarization models are typically pre-trained on large amounts of generic texts . large annotated datasets of reviews paired with reference summaries are not available . |
| Approach: | They propose a few-shot method which uses adapters to store in-domain knowledge . they pre-train adapters on unannotated customer reviews and fine-tune them on annotated datasets . |
| Outcome: | The proposed method can store in-domain knowledge and improves on large annotated reviews . it improves coherence and redundancies on the Amazon and Yelp datasets . |