Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for document summarization use graphs and unlabeled documents . Existing models require labeled data, and it is expensive to create summarized documents. |
| Approach: | They propose to rank sentences using transformer attentions and pre-training objectives by unlabeled documents. |
| Outcome: | The proposed model achieves state-of-the-art on unsupervised summarization and is less dependent on sentence positions. |
Similar Papers
Bipartite Graph Pre-training for Unsupervised Extractive Summarization with Graph Convolutional Auto-Encoders (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to rank sentences using pre-trained embeddings create a gap due to different optimization objectives. |
| Approach: | They propose a pre-trained embedding process that optimizes informative sentences . they use sentence-word bipartite graphs to model intra-sentential distinctive features . |
| Outcome: | The proposed model outperforms heavy BERT- or RoBERTa-based sentence ranking methods by providing summary-worthy representations. |
TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing abstractive summarization models ignore abundant unlabeled corpora resources . TED outperforms all unsupervised abstractive baselines on NYT, CNN/DM and English Gigaword datasets . |
| Approach: | They propose a transformer-based unsupervised text summarization system with pretraining on large-scale data. |
| Outcome: | The proposed system outperforms baseline models on NYT, CNN/DM and English Gigaword datasets with various document styles. |
Hierarchical Transformers for Multi-Document Summarization (P19-1)
Copied to clipboard
| Challenge: | Existing models for multidocument summarization have been developed that can process multiple documents in a hierarchical manner. |
| Approach: | They propose a neural summarization model which can process multiple input documents and distill Transformer architecture with the ability to encode documents in a hierarchical manner. |
| Outcome: | The proposed model improves on the WikiSum dataset and can process multiple documents in a hierarchical manner. |
Exploiting Discourse-Level Segmentation for Extractive Summarization (D19-54)
Copied to clipboard
| Challenge: | Existing approaches to extract summarize text are based on sentences as the elementary unit, but semantic segments containing supplementary information or descriptive details are often nonessential in the generated summaries. |
| Approach: | They propose to exploit discourse-level segmentation as a finer-grained means to more precisely pinpoint the core content in a document. |
| Outcome: | The proposed method improves extractive summarization performance on CNN/Daily Mail dataset. |
Improving Abstractive Dialogue Summarization with Hierarchical Pretraining and Topic Segment (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for meeting summary have limited the ability to deal with long-term dependency. |
| Approach: | They propose a hierarchical transformer encoder-decoder network with multi-task pre-training to capture key sentences at word level and generate them at word-level. |
| Outcome: | The proposed model is superior to the previous methods in meeting summary datasets AMI and ICSI. |
Neural Document Summarization by Jointly Learning to Score and Select Sentences (P18-1)
Copied to clipboard
| Challenge: | Sentence scoring and sentence selection are two main steps in extractive document summarization systems. |
| Approach: | They propose an end-to-end neural network framework for extractive document summarization by jointly learning to score and select sentences. |
| Outcome: | The proposed framework outperforms the state-of-the-art summarization models on the CNN/Daily Mail dataset. |
On Extractive and Abstractive Neural Document Summarization with Transformer Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | We present a method to produce abstractive summaries of documents that exceed several thousand words . we compare transformer based methods to extractive methods, but extractive models score higher . |
| Approach: | They propose a method to generate abstractive summaries of documents that exceed several thousand words via neural abstractive summary. |
| Outcome: | The proposed method produces abstractive summaries of documents that exceed several thousand words . it is compared with baseline methods, state-of-the-art models and variants of the proposed method . |
Self-Supervised Learning for Contextualized Extractive Summarization (P19-1)
Copied to clipboard
| Challenge: | Existing models for extractive summarization are usually trained from scratch with a cross-entropy loss . previous work builds an end-to-end system to learn to choose sentences without explicitly modeling document context . |
| Approach: | They propose three auxiliary pre-training tasks that learn to capture the document context in a self-supervised fashion. |
| Outcome: | The proposed models outperform existing models on a CNN/DM dataset. |
HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization (P19-1)
Copied to clipboard
| Challenge: | Neural extractive summarization models employ hierarchical encoders with inaccurate sentence-level labels. |
| Approach: | They propose a method to pre-train a hierarchical encoder with unlabeled data. |
| Outcome: | The proposed model outperforms its initialized counterpart by 1.25 ROUGE on CNN and 2.0 ROUGEE on a version of New York Times dataset. |
A Set Prediction Network For Extractive Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent approaches to extracting salient sentences from source document are naive and lack dependencies between sentences. |
| Approach: | They propose a set prediction network to detect redundancy relationship between sentences . they use a non-autoregressive decoder to predict sentences in parallel . |
| Outcome: | The proposed method outperforms previous state-of-the-art models on extracted summary datasets. |