Shashi Narayan, Ronald Cardenas, Nikos Papasarantopoulos, Shay B. Cohen, Mirella Lapata, Jiangsheng Yu, Yi Chang
| Challenge: | Document modeling is essential to a variety of natural language understanding tasks. |
| Approach: | They propose to use external information to improve document modeling for sentence extraction problems. |
| Outcome: | The proposed model outperforms baseline models on document summarization and answer selection tasks and achieves state-of-the-art results on WikiQA and NewsQA. |
Similar Papers
Neural Document Summarization by Jointly Learning to Score and Select Sentences (P18-1)
Copied to clipboard
| Challenge: | Sentence scoring and sentence selection are two main steps in extractive document summarization systems. |
| Approach: | They propose an end-to-end neural network framework for extractive document summarization by jointly learning to score and select sentences. |
| Outcome: | The proposed framework outperforms the state-of-the-art summarization models on the CNN/Daily Mail dataset. |
A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents (N18-2)
Copied to clipboard
Arman Cohan, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Seokhwan Kim, Walter Chang, Nazli Goharian
| Challenge: | Existing abstractive summarization models focus on summarizing sentences and short documents. |
| Approach: | They propose a hierarchical encoder that models the discourse structure of a document, and an attentive discourse-aware decoder to generate the summary. |
| Outcome: | The proposed model significantly outperforms state-of-the-art models on two large-scale datasets of scientific papers. |
Improving Abstractive Document Summarization with Salient Information Modeling (P19-1)
Copied to clipboard
| Challenge: | Abstractive document summarization is a task of natural language generation which generates fluent summaries with salient information automatically. |
| Approach: | They propose to incorporate a Gaussian focal bias on attention scores into an encoder to enhance the perception of local context and to distinguish salient information precisely. |
| Outcome: | The proposed framework outperforms state-of-the-art models on the CNN/Daily Mail benchmark and is based on a focus-attention mechanism and two new extensions. |
Attention Optimization for Abstractive Document Summarization (D19-1)
Copied to clipboard
| Challenge: | Abstractive summarization models require attention to reproduce the most salient information. |
| Approach: | They propose to use local and global variances to augment the vanilla attention model to reproduce the most salient information and avoid repetitions. |
| Outcome: | The proposed attention refinement unit can reproduce the most salient information and avoid repetitions on CNN/Daily Mail dataset. |
Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for document summarization use graphs and unlabeled documents . Existing models require labeled data, and it is expensive to create summarized documents. |
| Approach: | They propose to rank sentences using transformer attentions and pre-training objectives by unlabeled documents. |
| Outcome: | The proposed model achieves state-of-the-art on unsupervised summarization and is less dependent on sentence positions. |
Improving Neural Abstractive Document Summarization with Explicit Information Selection Modeling (D18-1)
Copied to clipboard
| Challenge: | Existing neural abstractive methods for document summarization are not effective for document summary. |
| Approach: | They propose to extend basic neural encoding-decoding framework with an information selection layer to explicitly model and optimize the information selection process in abstractive document summarization. |
| Outcome: | The proposed model outperforms state-of-the-art methods on document summarization tasks significantly. |
Frame Semantic-Enhanced Sentence Modeling for Sentence-level Extractive Text Summarization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Sentence-level extractive text summarization is difficult to model the importance of sentences. |
| Approach: | They propose a Frame Semantic-Enhanced Sentence Modeling for Extractive Summarization that leverages Frame semantics to model sentences from both intra-sentence level and inter-sentent level. |
| Outcome: | The proposed model outperforms six state-of-the-art methods on two benchmark corpus datasets. |
A Hierarchical Neural Attention-based Text Classifier (D18-1)
Copied to clipboard
| Challenge: | Existing hierarchical classification models are unable to handle large corpora and the number of categories increases with increasing corpus. |
| Approach: | They propose to use external knowledge to introduce a hierarchical neural attention-based classifier to help with the classification of documents. |
| Outcome: | The proposed model performs better than or comparable to state-of-the-art hierarchical models at significantly lower computational cost while maintaining high interpretability. |
Hie-BART: Document Summarization with Hierarchical BART (2021.naacl-srw)
Copied to clipboard
| Challenge: | Existing document summarization models do not capture hierarchical structures of documents . proposed model incorporates multi-granularity self-attention (MG-SA) |
| Approach: | They propose a new abstractive document summarization model, hierarchical BART . the proposed model captures hierarchically structured sentences in the BART model . |
| Outcome: | The proposed model outperforms baseline models and improves performance on CNN/Daily Mail dataset. |
A Mixed Hierarchical Attention Based Encoder-Decoder Approach for Standard Table Summarization (N18-2)
Copied to clipboard
| Challenge: | Structured data summarization involves generation of summaries from structured input data. |
| Approach: | They propose a hierarchical attention-based encoder-decoder model which leverages the structure in addition to the content of the tables. |
| Outcome: | The proposed model improves on the weathergov dataset by 30% over the current state-of-the-art. |