AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | State-of-the-art abstractive summarization models rely on extensive labeled data, which lowers their generalization ability on domains where such data are not available. |
| Approach: | They propose to use domain adaptation methods to simulate the low-resource domain adaptation setting for abstractive summarization systems with existing datasets across six diverse target domains. |
| Outcome: | The proposed model can be used to adapt to a low-resource domain adaptation setting. |
Similar Papers
Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsing (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in deep learning have enabled several approaches to successfully parse more complex queries, but these models require a large amount of annotated training data to parser on new domains (e.g. reminder, music). |
| Approach: | They propose a method that adapts task-oriented semantic parsers to low-resource domains and outperforms a supervised neural model at a 10-fold data reduction. |
| Outcome: | The proposed method outperforms baseline methods on a newly collected multi-domain task-oriented semantic parsing dataset (TOPv2) . |
Adversarial Domain Adaptation Using Artificial Titles for Abstractive Title Generation (P19-1)
Copied to clipboard
| Challenge: | Obtaining good quality labeled data can be difficult and expensive for abstractive summarization models . authors propose the use of artificial titles for unlabeled target documents . |
| Approach: | They propose to use artificial titles and sequential training to capture grammatical style of unlabeled target domains to adapt to/from news articles and Stack Exchange posts. |
| Outcome: | The proposed techniques can boost performance for unsupervised adaptation and fine-tuning with limited target data. |
A Guide To Effectively Leveraging LLMs for Low-Resource Text Summarization: Data Augmentation and Semi-supervised Approaches (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches for low-resource text summarization use large language models (LLMs) but such models suffer from inconsistent outputs and are difficult to adapt to domain-specific data. |
| Approach: | They propose two methods to effectively utilize large language models for low-resource text summarization. |
| Outcome: | The proposed methods synthesize high-quality documents using LLaMA-3-70b-Instruct model . they achieve competitive ROUGE scores as a fully supervised method with 5% of the labeled data. |
Taming Pre-trained Language Models with N-gram Representations for Low-Resource Domain Adaptation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to train pre-trained models require domain-specific data and computational resources. |
| Approach: | They propose a domain-aware N-gram Adaptor to incorporate unseen and domain-specific words into a generic pretrained model. |
| Outcome: | The proposed model can improve on eight low-resource tasks using limited data with lower computational costs. |
Learning Faster with Better Tokens: Parameter-Efficient Vocabulary Adaptation for Specialized Text Summarization (2026.acl-long)
Copied to clipboard
| Challenge: | a new approach to adapt generalist models to expert domains is needed to overcome this problem. |
| Approach: | They propose a parameter-efficient domain adaptation approach that combines vocabulary adaptation with pretraining for LLM-based text summarization. |
| Outcome: | The proposed approach reduces training time by 35-55% over continual pretraining and reduces parameter counts up to 37% w.r.t expansion-only methods. |
Word Matters: What Influences Domain Adaptation in Summarization? (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can generalize domain datasets unseen during training but are not able to predict domain adaptation performance. |
| Approach: | They propose to quantify dataset learning difficulty as the learning difficulty of generative summarization, which is determined by word-based compression rate and abstraction level. |
| Outcome: | The proposed model can predict performance on unknown domain datasets without training, and it is based on the findings. |
Pre-training for Abstractive Document Summarization by Reinstating Source Text (2020.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive document summarization models are often trained on limited supervised data . authors present three objectives for pretraining abstractive summarizing models . |
| Approach: | They propose to pre-train a SEQ2SEQ based abstractive summarization model on unlabeled text. |
| Outcome: | The proposed method improves on two benchmark summarization datasets with 19GB of text . the goal is sentence reordering, next sentence generation and masked document generation . |
Long Document Summarization in a Low Resource Setting using Pretrained Language Models (2021.acl-srw)
Copied to clipboard
Ahsaas Bajaj, Pavitra Dangati, Kalpesh Krishna, Pradhiksha Ashok Kumar, Rheeya Uppaal, Bradford Windsor, Eliot Brenner, Dominic Dotterrer, Rajarshi Das, Andrew McCallum
| Challenge: | Existing abstractive summarization methods only achieve 17.9 ROUGE-L in low-resource settings. |
| Approach: | They propose to use a modern abstractive summarization algorithm to extract salient sentences from long documents to improve their performance. |
| Outcome: | The proposed method beats several competitive salience detection baselines and the identified salient sentences agree with independent human labeling by domain experts. |
Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks (2020.acl-main)
Copied to clipboard
Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, Noah A. Smith
| Challenge: | Language models prerained on text from a wide variety of sources form the foundation of today’s NLP. |
| Approach: | They propose to tailor a pretrained model to the domain of a target task by using domain-adaptive pretraining in-domain. |
| Outcome: | The proposed model can be tailored to the domain of a target task and perform well under both high- and low-resource settings. |
Multi-Stage Pre-training for Low-Resource Domain Adaptation (2020.emnlp-main)
Copied to clipboard
Rong Zhang, Revanth Gangi Reddy, Md Arafat Sultan, Vittorio Castelli, Anthony Ferritto, Radu Florian, Efsun Sarioglu Kayi, Salim Roukos, Avi Sil, Todd Ward
| Challenge: | Existing approaches to transfer learning target data to in-domain text . prior work has adapted pre-trained LMs to specific domains . |
| Approach: | They extend the vocabulary of a pretrained language model with domain-specific terms to create synthetic tasks that help it transfer to downstream tasks. |
| Outcome: | The proposed approaches show significant performance gains on extractive reading comprehension, document ranking and duplicate question detection tasks. |