Papers with Gigaword
Classical Structured Prediction Losses for Sequence to Sequence Learning (N18-1)
Copied to clipboard
| Challenge: | Recent work on training neural attention models at the sequence level has focused on a series of objective functions commonly used for structured prediction. |
| Approach: | They propose to use objective functions commonly used to train linear models for structured prediction to train neural attention models at the sequence-level using either reinforcement learning-style methods or beam search optimization. |
| Outcome: | The proposed model outperforms beam search optimization on German-English translation and abstractive summarization tasks. |
Entity Commonsense Representation for Neural Abstractive Summarization (N18-1)
Copied to clipboard
| Challenge: | Current ELS’s are not sufficiently effective, possibly introducing unresolved ambiguities and irrelevant entities. |
| Approach: | They propose an off-the-shelf entity linking system to extract linked entities and propose Entity2Topic (E2T) module attachable to a sequence-to-sequence model that transforms a list of entities into a vector representation of the topic of the summary. |
| Outcome: | The proposed model improves the performance of the Gigaword and CNN summarization datasets by at least 2 ROUGE points. |
Avoiding Overlap in Data Augmentation for AMR-to-Text Generation (2021.acl-short)
Copied to clipboard
| Challenge: | Using unlabeled data to boost model performance is common practice in machine learning and natural language processing. |
| Approach: | They propose methods for excluding parts of Gigaword to remove overlap . they propose to use the AMR dataset for AMR-to-text generation . |
| Outcome: | The proposed approach leads to a more realistic evaluation of the task of AMR-to-text generation. |
Integrating Semantic Scenario and Word Relations for Abstractive Sentence Summarization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing graph-based methods only consider word relations or structure information, which neglect the correlation between them. |
| Approach: | They propose a Dual Graph network for Abstractive Sentence Summarization that captures word relations and structure information from sentences. |
| Outcome: | The proposed model outperforms state-of-the-art methods on two popular benchmark datasets. |
ProphetNet: Predicting Future N-gram for Sequence-to-SequencePre-training (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing sequence-to-sequence models are optimized for future n-gram prediction and n stream self-attention mechanism. |
| Approach: | They propose a self-supervised objective called future n-gram prediction and the proposed n stream self-attention mechanism to optimize the model for sequence-to-sequence learning. |
| Outcome: | The proposed model achieves state-of-the-art on CNN/DailyMail, Gigaword, and SQuAD 1.1 benchmarks compared to the models using the same scale pre-training corpus. |
Discrete Diffusion Language Model for Efficient Text Summarization (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing discrete diffusion models fail on conditional long-text generation due to incompatibility between the backbone architectures and the random noising process. |
| Approach: | They propose a semantic-aware noising process that enables Transformer backbones to handle long sequences effectively. |
| Outcome: | The proposed model outperforms existing models on three benchmark summarization datasets while achieving much faster inference speed compared to autoregressive models. |
Improving the Robustness of Summarization Systems with Dual Augmentation (2023.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that state-of-the-art summarization models have a significant decrease in performance on adversarial and noisy test sets. |
| Approach: | They propose a SummAttacker approach to generate adversarial samples based on pre-trained language models that can generate word-level synonym substitution and noise. |
| Outcome: | The proposed model performs better on noisy, attacked, and clean datasets than baseline models and is more robust on noisy and attacked datasets. |
Constructing a Lexicon of Relational Nouns (L18-1)
Copied to clipboard
| Challenge: | Existing systems for extracting relations expressed using nouns do not exist for relational noun. |
| Approach: | They contribute a lexicon of 6,224 labeled nouns which includes 1,446 relational noun. |
| Outcome: | The proposed classifier achieves 70.4% F1 on held out nouns among the most common 2,500 word types in Gigaword. |
Deep Reinforcement Learning with Distributional Semantic Rewards for Abstractive Summarization (D19-1)
Copied to clipboard
| Challenge: | Abstractive summarization tasks are often based on deep reinforcement learning (RL) but the traditional reward system Rouge-L simply looks for exact n-grams matches between candidates and annotated references, which makes the generated sentences repetitive and incoherent. |
| Approach: | They propose to use distributional semantics to measure matching degrees instead of Rouge-L to generate sentences with n-grams matches. |
| Outcome: | The proposed reward has superiority over the existing reward, despite the incoherence of the generated sentences. |
PALM: Pre-training an Autoencoding&Autoregressive Language Model for Context-conditioned Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing techniques for natural language understanding and generation use autoencoding and/or autoregressive objectives to train models. |
| Approach: | They propose a self-supervised pre-training scheme that pre-trains an autoencoding and autoregressive language model on a large unlabeled corpus for generating new text conditioned on context. |
| Outcome: | The proposed scheme achieves state-of-the-art results on a variety of language generation benchmarks covering generative question answering, abstractive summarization and conversational response generation. |
TeaForN: Teacher-Forcing with N-grams (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to address exposure bias and lack of differentiability in sequence generation models with teacherforcing have failed to address these issues. |
| Approach: | They propose a method that uses a stack of N decoders to decode along a secondary time axis and allows model-parameter updates based on N prediction steps. |
| Outcome: | Empirically, teaForN boosts generation quality on one Machine Translation benchmark, WMT 2014 English-French, and two News Summarization benchmarks, CNN/Dailymail and Gigaword. |
CiteSum: Citation Text-guided Scientific Extreme Summarization and Domain Adaptation with Limited Supervision (2022.emnlp-main)
Copied to clipboard
| Challenge: | Scientific extreme summarization (TLDR) aims to form ultra-short summaries of scientific papers . previous attempts failed to scale up due to heavy human annotation and domain expertise . |
| Approach: | They propose a method to automatically extract TLDR summaries from scientific papers . they propose 'citeSum' with no human annotation, which is 30 times larger than SciTLDR . |
| Outcome: | The proposed approach outperforms most fully-supervised methods on SciTLDR without fine-tuning and achieves state-of-the-art results with only 128 examples. |
Improving Faithfulness by Augmenting Negative Summaries from Fake Documents (2022.emnlp-main)
Copied to clipboard
| Challenge: | Current abstractive summarization systems tend to hallucinate unfaithful content . however, the most common method does not disentangle factual errors from other errors. |
| Approach: | They propose a back-translation-style approach to augment negative samples that mimic factual errors made by the model. |
| Outcome: | The proposed method improves faithfulness without sacrificing informativeness . it incorporates negative samples into training, and produces faithful/unfaithful summaries . |