| Challenge: | Abstractive summarization models are typically pre-trained on large amounts of generic texts . large annotated datasets of reviews paired with reference summaries are not available . |
| Approach: | They propose a few-shot method which uses adapters to store in-domain knowledge . they pre-train adapters on unannotated customer reviews and fine-tune them on annotated datasets . |
| Outcome: | The proposed method can store in-domain knowledge and improves on large annotated reviews . it improves coherence and redundancies on the Amazon and Yelp datasets . |
Similar Papers
Improving Zero and Few-Shot Abstractive Summarization with Intermediate Fine-tuning and Data Augmentation (2021.naacl-main)
Copied to clipboard
Alexander Fabbri, Simeng Han, Haoyuan Li, Haoran Li, Marjan Ghazvininejad, Shafiq Joty, Dragomir Radev, Yashar Mehdad
| Challenge: | Recent advances in summarization are driven by the availability of large datasets such as the CNN-DailyMail corpus and the New York Times corpus. |
| Approach: | They propose a method for fine-tuning pretrained models for summarization in unsupervised manner . they use Wikipedia data to produce pseudo-summaries which contain characteristics of target dataset . |
| Outcome: | The proposed method achieves state-of-the-art, zero-shot abstractive summarization performance on CNN-DailyMail dataset and compares with other methods on other datasets. |
Few-Shot Learning for Opinion Summarization (2020.emnlp-main)
Copied to clipboard
| Challenge: | a recent study shows that abstractive summarization models fail to capture their essential properties due to the high cost of summary production. |
| Approach: | They propose a few-shot framework for abstractive opinion summarization that bootstraps the output of an unsupervised model. |
| Outcome: | The proposed framework outperforms extractive and abstractive methods on Amazon and Yelp datasets. |
Few-shot fine-tuning SOTA summarization models for medical dialogues (2022.naacl-srw)
Copied to clipboard
| Challenge: | Abstractive summarization of medical dialogues is a challenge for standard training approaches due to the paucity of suitable datasets. |
| Approach: | They propose to use medical dialogues to generate abstractive summaries using transformer-based models with zero-shot and few-shot learning strategies. |
| Outcome: | The proposed models were compared with a medical dialogue dataset with 143 snippets and a general domain and dialogue-specific text to assess their performance. |
PSP: Pre-trained Soft Prompts for Few-Shot Abstractive Summarization (2022.coling-1)
Copied to clipboard
| Challenge: | Experimental results show that our method outperforms full-model tuning in few-shot abstractive summarization tasks. |
| Approach: | They propose a soft prompts architecture with prompt pre-training and prompt fine-tuning paradigm to support few-shot abstractive summarization. |
| Outcome: | The proposed model outperforms Prompt Tuning and Profix-Tuning on CNN/DailyMail and XSum datasets and outperfies Profix Tuning by a large margin. |
Leveraging Pretrained Models for Automatic Summarization of Doctor-Patient Conversations (2021.findings-emnlp)
Copied to clipboard
Longxiang Zhang, Renato Negrinho, Arindam Ghosh, Vasudevan Jagannathan, Hamid Reza Hassanzadeh, Thomas Schaaf, Matthew R. Gormley
| Challenge: | Using pretrained transformer models for automatically summarizing doctor-patient conversations presents challenges . limited training data, domain shift, long and noisy transcripts, and high target summary variability are challenges compared to human annotators. |
| Approach: | They propose a method for fine-tuning pretrained transformer models for automatically summarizing doctor-patient conversations directly from transcripts. |
| Outcome: | The proposed method surpasses the performance of an average human annotator and the quality of previous published work for the task. |
FewshotQA: A simple framework for few-shot learning of question answering tasks using pre-trained text-to-text models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-trained models need fine-tuning on tens of thousands of examples to achieve good results. |
| Approach: | They propose a framework that leverages pre-trained text-to-text models and aligns them with their pre-training framework. |
| Outcome: | The proposed framework outperforms the XLM-Roberta-large on multiple QA benchmarks and is applicable to multilingual situations. |
Make The Most of Prior Data: A Solution for Interactive Text Summarization with Preference Feedback (2022.findings-naacl)
Copied to clipboard
Duy-Hung Nguyen, Nguyen Viet Dung Nghiem, Bao-Sinh Nguyen, Dung Tien Tien Le, Shahab Sabahi, Minh-Tien Nguyen, Hung Le
| Challenge: | a framework to train summarization models with preference feedback is proposed . human-in-the-loop (HITL) allows humans to actively participate in supervising AI systems . |
| Approach: | They propose a framework to train summarization models with preference feedback interactively. |
| Outcome: | The proposed framework improves ROUGE scores and sample-efficiency in active, few-shot and online settings. |
Abstractive Document Summarization with Summary-length Prediction (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing abstractive summarization models do not consider summarizing-specific information such as the target summary length. |
| Approach: | They propose a method for enabling a model to understand summarization-specific information by predicting the summary length in the encoder and generating a summary of the predicted length in fine-tuning. |
| Outcome: | The proposed method improves ROUGE scores on the WikiHow, NYT, and CNN/DM datasets. |
Prompt-free and Efficient Few-shot Learning with Language Models (2022.acl-long)
Copied to clipboard
Rabeeh Karimi Mahabadi, Luke Zettlemoyer, James Henderson, Lambert Mathias, Marzieh Saeidi, Veselin Stoyanov, Majid Yazdani
| Challenge: | Existing methods for few-shot fine-tuning of pretrained language models require carefully engineered prompts and verbalizers to convert inputs into a cloze-format that the PLM can score. |
| Approach: | They propose a method for few-shot fine-tuning of pretrained language models that uses task-specific adapters instead of manually engineered prompts and verbalizers. |
| Outcome: | The proposed method outperforms existing state-of-the-art methods on a wide range of few shot NLP tasks. |
Fine-tuning Encoders for Improved Monolingual and Zero-shot Polylingual Neural Topic Modeling (2021.naacl-main)
Copied to clipboard
| Challenge: | Topic models can augment or replace bag-of-words inputs with pre-trained transformer-based word prediction models. |
| Approach: | They propose several methods for fine-tuning encoders to improve both monolingual and zero-shot polylingual topic modeling. |
| Outcome: | The proposed methods improve both monolingual and zero-shot polylingual topic modeling. |