| Challenge: | Abstractive summarization models have been gaining popularity, but performance of unsupervised models still lags behind supervised models. |
| Approach: | They propose to re-rank summary candidates in an unsupervised manner to close the performance gap between unsupervised and supervised models. |
| Outcome: | The proposed model improves unsupervised models by up to 7.27% and ChatGPT by up 6.86% relative mean ROUGE across four widely-adopted summarization benchmarks. |
Similar Papers
SummaReranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarization (2022.acl-long)
Copied to clipboard
| Challenge: | Sequence-to-sequence neural networks have enabled great progress in abstractive summarization. |
| Approach: | They propose to train a second-stage model performing re-ranking on a set of summary candidates by using a mixture of experts. |
| Outcome: | The proposed model outperforms the base model on CNN- DailyMail, XSum and Reddit TIFU with a base PEGASUS. |
Balancing Lexical and Semantic Quality in Abstractive Summarization (2023.acl-short)
Copied to clipboard
| Challenge: | Existing methods to reduce exposure bias in sequence-to-sequence models are underexplored. |
| Approach: | They propose a method to re-rank sequence-to-sequence neural models to reduce exposure bias. |
| Outcome: | The proposed method achieves an 89.67 BERTScore on the CNN/DailyMail and XSum datasets. |
Towards Summary Candidates Fusion (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for abstractive summarization are limited by the quality of the first-stage candidates. |
| Approach: | They propose a method that fuses several summary candidates to produce a novel abstractive second-stage summary. |
| Outcome: | The proposed method improves ROUGE scores and qualitative properties of fused summaries on several summarization datasets. |
Revisiting Zero-Shot Abstractive Summarization in the Era of Large Language Models from the Perspective of Position Bias (2024.naacl-short)
Copied to clipboard
| Challenge: | Position bias is a tendency of a model unfairly prioritizing information from certain parts of the input text over others, leading to undesirable behavior. |
| Approach: | They propose to measure position bias in large language models for zero-shot summarization tasks by measuring position bias. |
| Outcome: | The proposed model performance and position biases lead to new insights and discussion on zero-shot summarization tasks. |
RepSum: Unsupervised Dialogue Summarization based on Replacement Strategy (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to learn vital information from dialogue context with limited data are limited due to limited words in utterances and huge gap between dialogue and its summary. |
| Approach: | They propose an unsupervised strategy to learn vital information from dialogue context . the proposed model uses a hypothetical foundation that a superior summary approximates a replacement of the original dialogue . |
| Outcome: | The proposed model outperforms existing models on a number of datasets. |
SummEval: Re-evaluating Summarization Evaluation (2021.tacl-1)
Copied to clipboard
Alexander R. Fabbri, Wojciech Kryściński, Bryan McCann, Caiming Xiong, Richard Socher, Dragomir Radev
| Challenge: | a lack of comprehensive studies on evaluation metrics for text summarization hinders progress . a new study aims to improve evaluation metrics that correlate with human judgments . |
| Approach: | They propose to re-evaluate automatic evaluation metrics and share a toolkit for evaluation . they hope to promote a more complete evaluation protocol for text summarization . |
| Outcome: | The proposed evaluation metrics are inconsistent with existing evaluation protocols. |
UniSumm and SummZoo: Unified Model and Diverse Benchmark for Few-Shot Summarization (2023.acl-long)
Copied to clipboard
| Challenge: | a new benchmark summarization model is being developed to train few-shot summarizers . a large number of summarizing tasks are required to perform well in heterogeneous datasets. |
| Approach: | They propose a few-shot summarization model pre-trained with multiple summarizing tasks . they propose 'uniSumm' to be prefix-tuned to excel at any few-shot summarisation task . |
| Outcome: | The proposed model outperforms baseline models under automatic and human evaluations and achieves comparable results in human evaluation. |
Improving Factuality of Abstractive Summarization without Sacrificing Summary Quality (2023.acl-short)
Copied to clipboard
| Challenge: | Recent studies have shown that most abstractive summarization models are unfaithful and suffer from a wide range of hallucination. |
| Approach: | They propose a candidate summary generation and ranking technique to improve summary factuality without sacrificing quality. |
| Outcome: | The proposed method shows that the model trained using the proposed method improves on factuality and similarity-based metrics without conflicting with the model. |
On the Role of Supervision in Unsupervised Constituency Parsing (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on unsupervised constituency parsing uses labeled examples for tuning . a few-shot parser with labeles can outperform other approaches by a significant margin . |
| Approach: | They propose to use as few labeled examples as possible for model development . they propose to train existing models on the same labeles they access . |
| Outcome: | The proposed model outperforms other models on the WSJ development set by a significant margin . the proposed model can be further improved by augmentation and self-training . |
USB: A Unified Summarization Benchmark Across Tasks and Domains (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing summarization benchmarks lack the rich annotations needed to address important problems related to control and reliability. |
| Approach: | They propose a Wikipedia-derived summarization benchmark with crowd-sourced annotations . they find that fine-tuned models outperform larger few-shot prompted language models . |
| Outcome: | The proposed model outperforms many-shot prompted language models on multiple tasks . the proposed model is based on Wikipedia annotations and can be used in other domains . |