| Challenge: | Existing attention mechanisms for abstractive sentence summarization are based on rule-based methods and large-scale training corpora. |
| Approach: | They propose a contrastive attention mechanism that extends the sequence-to-sequence framework for abstractive sentence summarization task. |
| Outcome: | The proposed mechanism improves the state-of-the-art on the abstractive sentence summarization task. |
Similar Papers
Attention Optimization for Abstractive Document Summarization (D19-1)
Copied to clipboard
| Challenge: | Abstractive summarization models require attention to reproduce the most salient information. |
| Approach: | They propose to use local and global variances to augment the vanilla attention model to reproduce the most salient information and avoid repetitions. |
| Outcome: | The proposed attention refinement unit can reproduce the most salient information and avoid repetitions on CNN/Daily Mail dataset. |
A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents (N18-2)
Copied to clipboard
Arman Cohan, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Seokhwan Kim, Walter Chang, Nazli Goharian
| Challenge: | Existing abstractive summarization models focus on summarizing sentences and short documents. |
| Approach: | They propose a hierarchical encoder that models the discourse structure of a document, and an attentive discourse-aware decoder to generate the summary. |
| Outcome: | The proposed model significantly outperforms state-of-the-art models on two large-scale datasets of scientific papers. |
SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization (2021.acl-short)
Copied to clipboard
| Challenge: | Experimental results show that SimCLS can improve existing top-performing models by a large margin. |
| Approach: | They propose a framework for abstractive summarization that is conceptually simple and empirically powerful. |
| Outcome: | The proposed framework improves the performance of top-performing models by a large margin against existing top-scoring systems. |
Structure-Infused Copy Mechanisms for Abstractive Summarization (C18-1)
Copied to clipboard
| Challenge: | Experimental results show that system summaries struggle to preserve syntactic meaning of source texts. |
| Approach: | They propose to incorporate syntactic information from source sentences into abstractive summaries by structure-infused copy mechanisms. |
| Outcome: | The proposed approach compares favorably to state-of-the-art methods. |
Highlight-Transformer: Leveraging Key Phrase Aware Attention to Improve Abstractive Multi-Document Summarization (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing models do not consider key phrases in determining attention weights of self-attention . Existing work does not consider the importance of key phrases when determining weights . |
| Approach: | They propose a model with highlighting mechanism to assign greater attention weights to key phrases . they propose two structures of highlighting attention for each head and the multihead highlighting . experimental results show that their proposed model significantly outperforms the baseline model . |
| Outcome: | The proposed model outperforms the baseline models on a multi-news dataset. |
Self-Attention Guided Copy Mechanism for Abstractive Summarization (2020.acl-main)
Copied to clipboard
| Challenge: | Abstractive summarization models have been widely used to extract words from source into summary, but how to ensure that important words in source are copied remains a challenge. |
| Approach: | They propose a Transformer-based model to enhance copy mechanism by identifying the importance of each source word based on the degree centrality. |
| Outcome: | The proposed model outperforms baseline methods on CNN/Daily Mail and Gigaword datasets. |
A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss (P18-1)
Copied to clipboard
| Challenge: | extractive models can obtain sentence-level attention with high ROUGE scores but less readable. abstractive models generate novel words and phrases not copied from the source text. |
| Approach: | They propose to combine extractive and abstractive models to achieve a unified model that generates readable paragraphs with word-level attention. |
| Outcome: | The proposed model achieves state-of-the-art ROUGE scores while being the most informative and readable summarization on the CNN/Daily Mail dataset in a solid human evaluation. |
Topic-Aware Contrastive Learning for Abstractive Dialogue Summarization (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to abstractly summarize dialogues are limited to two or more interlocutors. |
| Approach: | They propose to use existing document summarization models to capture the various topic information of a conversation and outline salient facts for the captured topics. |
| Outcome: | The proposed method significantly outperforms baselines and achieves new state-of-the-art performance on benchmark datasets. |
Interpretable Multi-headed Attention for Abstractive Summarization at Controllable Lengths (2020.coling-main)
Copied to clipboard
| Challenge: | Abstractive summarization at controllable lengths is a challenging task in natural language processing . high variance in screen-sizes often require extensive human supervision to perform these modifications. |
| Approach: | They propose a supervised method to construct abstractive summaries of a text document at controllable lengths using an interpretable multi-headed attention mechanism. |
| Outcome: | The proposed method outperforms baselines on two low-resource datasets in English by 14.70%. |
Leveraging Locality in Abstractive Text Summarization (2022.emnlp-main)
Copied to clipboard
Yixin Liu, Ansong Ni, Linyong Nan, Budhaditya Deb, Chenguang Zhu, Ahmed Hassan Awadallah, Dragomir Radev
| Challenge: | Neural attention models have improved on many natural language processing tasks, but their quadratic memory complexity hinders their applications in long text summarization. |
| Approach: | They propose to use a restricted context to study locality in text summarization . they propose to employ a quadratic memory growth with respect to the input length . |
| Outcome: | The proposed model has better performance than baseline models with efficient attention modules. |