Friendly Topic Assistant for Transformer Based Abstractive Summarization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive document summarization is a comprehensive task in natural language processing. |
| Approach: | They propose a topic assistant that rearranges and learns document semantics . they propose TA that is compatible with Transformer-based models and user-friendly . |
| Outcome: | The proposed model is compatible with Transformer-based models and user-friendly. |
Similar Papers
Topic-Controllable Summarization: Topic-Aware Evaluation and Transformer Methods (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for topic-controllable summarization are limited by their recurrent architectures and require modifications to the model's architecture for controlling the topic. |
| Approach: | They propose a new topic-oriented evaluation measure to automatically evaluate the generated summaries based on the topic affinity between the generated summary and the desired topic. |
| Outcome: | The proposed method achieves better performance compared to more complicated embedding-based approaches while also being significantly faster. |
On Extractive and Abstractive Neural Document Summarization with Transformer Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | We present a method to produce abstractive summaries of documents that exceed several thousand words . we compare transformer based methods to extractive methods, but extractive models score higher . |
| Approach: | They propose a method to generate abstractive summaries of documents that exceed several thousand words via neural abstractive summary. |
| Outcome: | The proposed method produces abstractive summaries of documents that exceed several thousand words . it is compared with baseline methods, state-of-the-art models and variants of the proposed method . |
Enriching and Controlling Global Semantics for Text Summarization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive summarization models have been proven effective in creating fluent and informative summaries, but they suffer from the short-range dependency problem, causing them to produce summary that miss the key points of document. |
| Approach: | They propose a neural topic model empowered with normalizing flow to capture global semantics of the document and integrate them into the summarization model. |
| Outcome: | The proposed model outperforms state-of-the-art summarization models on five common text summarizing datasets, namely CNN/DailyMail, XSum, Reddit TIFU, arXiv, and PubMed. |
Improving Multi-Document Summarization through Referenced Flexible Extraction with Credit-Awareness (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to Multi-document summarization are limited due to the extremely long input length. |
| Approach: | They propose an extract-then-abstract Transformer framework to overcome the problem . they leverage pre-trained language models to construct hierarchical extractors and abstractors . |
| Outcome: | The proposed framework outperforms baseline models with comparable model sizes and achieves the best results on the Multi-News, Multi-XScience, and WikiCatSum corpora. |
Highlight-Transformer: Leveraging Key Phrase Aware Attention to Improve Abstractive Multi-Document Summarization (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing models do not consider key phrases in determining attention weights of self-attention . Existing work does not consider the importance of key phrases when determining weights . |
| Approach: | They propose a model with highlighting mechanism to assign greater attention weights to key phrases . they propose two structures of highlighting attention for each head and the multihead highlighting . experimental results show that their proposed model significantly outperforms the baseline model . |
| Outcome: | The proposed model outperforms the baseline models on a multi-news dataset. |
Flight of the PEGASUS? Comparing Transformers on Few-shot and Zero-shot Multi-document Abstractive Summarization (2020.coling-main)
Copied to clipboard
| Challenge: | Recent work has shown that pre-trained transformers obtain remarkable performance on many natural language processing tasks including automatic summarization. |
| Approach: | They propose to use transformers to generate multi-document summarization where the summary is explicitly conditioned on a user-given topic statement or question. |
| Outcome: | The proposed models perform well on four challenging summarization datasets from the general domain and one from consumer health. |
Improving Abstractive Document Summarization with Salient Information Modeling (P19-1)
Copied to clipboard
| Challenge: | Abstractive document summarization is a task of natural language generation which generates fluent summaries with salient information automatically. |
| Approach: | They propose to incorporate a Gaussian focal bias on attention scores into an encoder to enhance the perception of local context and to distinguish salient information precisely. |
| Outcome: | The proposed framework outperforms state-of-the-art models on the CNN/Daily Mail benchmark and is based on a focus-attention mechanism and two new extensions. |
Hierarchical Transformers for Multi-Document Summarization (P19-1)
Copied to clipboard
| Challenge: | Existing models for multidocument summarization have been developed that can process multiple documents in a hierarchical manner. |
| Approach: | They propose a neural summarization model which can process multiple input documents and distill Transformer architecture with the ability to encode documents in a hierarchical manner. |
| Outcome: | The proposed model improves on the WikiSum dataset and can process multiple documents in a hierarchical manner. |
Topic-Guided Abstractive Multi-Document Summarization (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on multi-document summarization (MDS) focus on extractive and abstractive approaches to create a fluent and concise summary for a collection of thematically related documents. |
| Approach: | They propose a novel abstractive MDS model that represents multiple documents as a heterogeneous graph and then applies a graph-to-sequence framework to generate summaries. |
| Outcome: | The proposed model outperforms state-of-the-art models on Rouge scores and human evaluation, while learning high-quality topics. |
Enriching Transformers with Structured Tensor-Product Representations for Abstractive Summarization (2021.naacl-main)
Copied to clipboard
Yichen Jiang, Asli Celikyilmaz, Paul Smolensky, Paul Soulos, Sudha Rao, Hamid Palangi, Roland Fernandez, Caitlin Smith, Mohit Bansal, Jianfeng Gao
| Challenge: | Abstractive summarization is the task of generating a concise summary of input documents . a middle-aged man and a young girl died after they were unable to avoid the plane . |
| Approach: | They propose a model that enriches the original Transformer with a Tensor Product Representation for abstractive summarization. |
| Outcome: | The proposed model outperforms the Transformer and the original TP-Transformer significantly on several datasets. |