Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarization (2022.acl-long)
Copied to clipboard
| Challenge: | Text summarization aims to generate a short summary for an input text. |
| Approach: | They propose a non-autoregressive unsupervised summarization approach which performs edit-based search towards a heuristicically defined score and generates a summary as pseudo-groundtruth. |
| Outcome: | The proposed approach achieves state-of-the-art performance for unsupervised summarization, while improving inference efficiency. |
Similar Papers
Directed Acyclic Transformer Pre-training for High-quality Non-autoregressive Text Generation (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing non-AutoRegressive (NAR) text generation models lack proper pre-training, making them far behind pre-trained autoregressive models. |
| Approach: | They propose a novel pre-training task to promote prediction consistency in non-autoregressive (NAR) generation. |
| Outcome: | The proposed model outperforms existing pre-trained models and achieves 17 times speedup in throughput. |
Non-Autoregressive Sentence Ordering (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing sentence ordering approaches only leverage unilateral dependencies during decoding and cannot fully explore the semantic dependency between sentences. |
| Approach: | They propose a non-autoregressive ordering network that explores bilateral dependencies between sentences and predicts sentences for each position in parallel. |
| Outcome: | The proposed model outperforms existing autoregressive sentence ordering approaches and yields competitive performance compared with the state-of-the-arts. |
Non-Autoregressive Text Generation with Pre-trained Language Models (2021.eacl-main)
Copied to clipboard
| Challenge: | Autoregressive generation models generate tokens in a left-to-right, token-by-token fashion, resulting in lag in inference. |
| Approach: | They propose to use BERT as the backbone of a non-autoregressive generation model for greatly improved performance. |
| Outcome: | The proposed model outperforms existing non-autoregressive models and achieves competitive performance with many strong autoregressive model. |
Simple Unsupervised Summarization by Contextual Matching (P19-1)
Copied to clipboard
| Challenge: | Existing methods for sentence summarization require a large amount of parallel data for supervision to work. |
| Approach: | They propose an unsupervised method for sentence summarization using only language modeling. |
| Outcome: | The proposed method maintains continuous contextual matching while maintaining output fluency without any paired examples. |
TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing abstractive summarization models ignore abundant unlabeled corpora resources . TED outperforms all unsupervised abstractive baselines on NYT, CNN/DM and English Gigaword datasets . |
| Approach: | They propose a transformer-based unsupervised text summarization system with pretraining on large-scale data. |
| Outcome: | The proposed system outperforms baseline models on NYT, CNN/DM and English Gigaword datasets with various document styles. |
JANUS: Joint Autoregressive and Non-autoregressive Training with Auxiliary Loss for Sequence Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to train autoregressive and non-autoregressive models only consider relevance of model parameters, ignoring correlations between the two manners. |
| Approach: | They propose a joint autoregressive and non-autoregressive training method using aUxiliary losS to enhance the model performance in both AR and NAR manners simultaneously. |
| Outcome: | The proposed method improves the model performance in both AR and NAR manners and reduces the inference latency. |
Unsupervised Extractive Opinion Summarization Using Sparse Coding (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for opinion summarization rely on human annotations, which may not be feasible. |
| Approach: | They propose to perform opinion summarization in an unsupervised manner by using a dictionary learning algorithm that implicitly captures semantic information from the review text. |
| Outcome: | The proposed algorithm performs well on SPACE and AMAZON datasets and performs controllable summarization to generate aspect-specific summaries using only a few samples. |
Generating Multiple-Length Summaries via Reinforcement Learning for Unsupervised Sentence Summarization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models to summarize texts without ground-truth summaries are extractive, which remove words from texts and thus are less flexible than abstractive models. |
| Approach: | They propose an unsupervised model that extracts words from texts and makes them mutually enhance each other. |
| Outcome: | The proposed model outperforms both abstractive and extractive models, while generating new words not contained in input texts. |
Unsupervised Opinion Summarization with Noising and Denoising (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for abstractive summarization are limited and cannot be easily sourced. |
| Approach: | They propose a supervised learning model which learns to denoise the input and generate original reviews. |
| Outcome: | The proposed model improves on the baselines of abstractive and extractive models on a large dataset with only a few reviews and no ground truth summaries. |
Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for document summarization use graphs and unlabeled documents . Existing models require labeled data, and it is expensive to create summarized documents. |
| Approach: | They propose to rank sentences using transformer attentions and pre-training objectives by unlabeled documents. |
| Outcome: | The proposed model achieves state-of-the-art on unsupervised summarization and is less dependent on sentence positions. |