InheritSumm: A General, Versatile and Compact Summarizer by Distilling from GPT (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in large language models have revolutionized the way summarization is generated. |
| Approach: | They propose a summarization model derived from GPT-3.5 through distillation that is compact and has comparable summarizing capabilities to GPT-3. |
| Outcome: | The proposed model outperforms the established best small models in prefix-tuning and full-data fine-tuned scenarios. |
Similar Papers
UniSumm and SummZoo: Unified Model and Diverse Benchmark for Few-Shot Summarization (2023.acl-long)
Copied to clipboard
| Challenge: | a new benchmark summarization model is being developed to train few-shot summarizers . a large number of summarizing tasks are required to perform well in heterogeneous datasets. |
| Approach: | They propose a few-shot summarization model pre-trained with multiple summarizing tasks . they propose 'uniSumm' to be prefix-tuned to excel at any few-shot summarisation task . |
| Outcome: | The proposed model outperforms baseline models under automatic and human evaluations and achieves comparable results in human evaluation. |
Impossible Distillation for Paraphrasing and Summarization: How to Make High-quality Lemonade out of Small, Low-quality Model (2024.naacl-long)
Copied to clipboard
Jaehun Jung, Peter West, Liwei Jiang, Faeze Brahman, Ximing Lu, Jillian Fisher, Taylor Sorensen, Yejin Choi
| Challenge: | Impossible Distillation is a framework for paraphrasing and sentence summarization that can be trained from a low-quality teacher model. |
| Approach: | They propose a framework that distills a high-quality dataset from a low-quality teacher . they hypothesize and verify the paraphrastic proximity intrinsic to pre-trained LMs . |
| Outcome: | The proposed framework outperforms baseline models on unconstrained paraphrase generation and sentence summarization benchmarks. |
Efficient Few-Shot Fine-Tuning for Opinion Summarization (2022.findings-naacl)
Copied to clipboard
| Challenge: | Abstractive summarization models are typically pre-trained on large amounts of generic texts . large annotated datasets of reviews paired with reference summaries are not available . |
| Approach: | They propose a few-shot method which uses adapters to store in-domain knowledge . they pre-train adapters on unannotated customer reviews and fine-tune them on annotated datasets . |
| Outcome: | The proposed method can store in-domain knowledge and improves on large annotated reviews . it improves coherence and redundancies on the Amazon and Yelp datasets . |
AlignSum: Data Pyramid Hierarchical Fine-tuning for Aligning with Human Summarization Preference (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Text summarization tasks employ Pre-trained Language Models (PLMs) to fit diverse datasets. |
| Approach: | They propose a human summarization preference alignment framework to align PLMs with human preferences. |
| Outcome: | The proposed framework narrows the gap between automatic and human evaluations by integrating three components. |
A Guide To Effectively Leveraging LLMs for Low-Resource Text Summarization: Data Augmentation and Semi-supervised Approaches (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches for low-resource text summarization use large language models (LLMs) but such models suffer from inconsistent outputs and are difficult to adapt to domain-specific data. |
| Approach: | They propose two methods to effectively utilize large language models for low-resource text summarization. |
| Outcome: | The proposed methods synthesize high-quality documents using LLaMA-3-70b-Instruct model . they achieve competitive ROUGE scores as a fully supervised method with 5% of the labeled data. |
PreSumm: Predicting Summarization Performance Without Summarizing (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in summarization models do not produce all documents in the same way, despite their inherent design principles and operational mechanisms. |
| Approach: | They propose a task where a system predicts summarization performance based solely on the source document. |
| Outcome: | The proposed task identifies documents that require manual summarization and improves dataset quality by filtering outliers and noisy documents. |
AUTOSUMM: Automatic Model Creation for Text Summarization (2021.emnlp-main)
Copied to clipboard
Sharmila Reddy Nangi, Atharv Tyagi, Jay Mundra, Sagnik Mukherjee, Raj Snehal, Niyati Chhaya, Aparna Garimella
| Challenge: | Recent efforts to develop deep learning models for text generation tasks are challenging for non-experts. |
| Approach: | They propose methods to automatically create deep learning models for extractive and abstractive summarization tasks using large language models. |
| Outcome: | The proposed methods achieve near state-of-the-art performance on a range of datasets. |
Summarizing, Simplifying, and Synthesizing Medical Evidence using GPT-3 (with Varying Success) (2023.acl-short)
Copied to clipboard
| Challenge: | Large language models are capable of producing high quality summaries of general domain news articles in few- and zero-shot settings, but it is unclear whether they are similarly capable in more specialized domains such as biomedicine. |
| Approach: | They use GPT-3 to generate single- and multi-document summaries of biomedical articles, given no supervision, using a set of annotations. |
| Outcome: | The proposed model outperforms fully supervised models in generic news summarization, but struggles to synthesize evidence across multiple documents. |
HydraSum: Disentangling Style Features in Text Summarization with Multi-Decoder Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive summarization systems implicitly encode “decisions” about summary properties, but these are not enforced. |
| Approach: | They propose a new summarization architecture that extends existing models to a mixture-of-experts version with multiple decoders. |
| Outcome: | The proposed architecture outperforms baseline models in obtaining stylistically-diverse summaries by sampling from individual decoders or their mixtures. |
GSum: A General Framework for Guided Neural Abstractive Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Abstractive summarization models are flexible, but they can be difficult to control. |
| Approach: | They propose a general and extensible guided summarization framework that takes different kinds of guidance as input and perform experiments across different varieties. |
| Outcome: | The proposed framework can generate more faithful summaries and different types of guidance generate qualitatively different summary. |