| Challenge: | Existing methods for sentence summarization require a large amount of parallel data for supervision to work. |
| Approach: | They propose an unsupervised method for sentence summarization using only language modeling. |
| Outcome: | The proposed method maintains continuous contextual matching while maintaining output fluency without any paired examples. |
Similar Papers
Discrete Optimization for Unsupervised Sentence Summarization with Word-Level Extraction (2020.acl-main)
Copied to clipboard
| Challenge: | Sentence summarization systems that use latent space to reconstruct the source sentence are unwillingly exploited. |
| Approach: | They propose a method that uses language modeling and semantic similarity metrics to find a high-scoring summary. |
| Outcome: | The proposed method achieves state-of-the-art for unsupervised sentence summarization according to ROUGE scores. |
BottleSum: Unsupervised and Self-supervised Sentence Summarization using the Information Bottleneck Principle (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to extractive and abstractive summarization rely on large-scale parallel corpora of input text and output summaries for direct supervision. |
| Approach: | They propose an unsupervised approach to sentence summarization using the Information Bottleneck principle. |
| Outcome: | The proposed method outperforms unsupervised models on automatic metrics and human evaluation along multiple attributes. |
Extractive Summarization as Text Matching (2020.acl-main)
Copied to clipboard
| Challenge: | Currently, most of the neural extractive summarization systems score and extract sentences individually and model the relationship between sentences. |
| Approach: | They propose to instantiate a neural extractive summarization task as a semantic text matching problem and use it to match a source document and candidate summaries in a semantic space. |
| Outcome: | The proposed framework is faster and more efficient than existing frameworks. |
Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for document summarization use graphs and unlabeled documents . Existing models require labeled data, and it is expensive to create summarized documents. |
| Approach: | They propose to rank sentences using transformer attentions and pre-training objectives by unlabeled documents. |
| Outcome: | The proposed model achieves state-of-the-art on unsupervised summarization and is less dependent on sentence positions. |
Relational Summarization for Corpus Analysis (N18-1)
Copied to clipboard
| Challenge: | Existing methods for summarizing textual content are often ignored . relationshipal questions are ubiquitous and varied. |
| Approach: | They propose a method which generates a natural language summary of the relationship between two lexical items in a corpus without reference to a knowledge base. |
| Outcome: | The proposed method generates a natural language summary of the relationship between two lexical items in a corpus without reference to a knowledge base. |
Abstractive Document Summarization without Parallel Data (2020.lrec-1)
Copied to clipboard
| Challenge: | Abstractive summarization typically relies on large collections of paired articles and summaries. |
| Approach: | They propose a system that relies only on example summaries and non-matching articles . they use an unsupervised sentence extractor that selects salient sentences . |
| Outcome: | The proposed system performs well on CNN/DailyMail benchmark and automatic generating a press release from a scientific journal article. |
The Summary Loop: Learning to Write Abstractive Summaries Without Examples (2020.acl-main)
Copied to clipboard
| Challenge: | Unsupervised abstractive summarization is important for news headlines and research papers . a novel method that encourages the inclusion of key terms from the original document into the summary is presented . |
| Approach: | They propose a method that encourages the inclusion of key terms from the original document into the summary by a coverage model along with a fluency model. |
| Outcome: | The proposed method outperforms existing methods on news summarization datasets and is competitive with existing methods. |
Unsupervised Learning of Sentence Embeddings Using Compositional n-Gram Features (N18-1)
Copied to clipboard
| Challenge: | Currently, unsupervised word embeddings are routinely trained on large amounts of raw text data. |
| Approach: | They propose to use unsupervised word embeddings to train distributed representations of sentences. |
| Outcome: | The proposed method outperforms state-of-the-art models on most benchmark tasks and is robust to the produced general-purpose sentence embeddings. |
SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization (2022.tacl-1)
Copied to clipboard
| Challenge: | Recent studies have shown that even state-of-the-art pre-trained language models can generate inconsistent summaries in more than 70% of all cases. |
| Approach: | They propose a method that enables NLI models to be used for inconsistency detection by segmenting documents into sentence units and aggregating scores between pairs of sentences. |
| Outcome: | The proposed method achieves state-of-the-art accuracy of 74.4% on six large inconsistency detection datasets. |
Bipartite Graph Pre-training for Unsupervised Extractive Summarization with Graph Convolutional Auto-Encoders (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to rank sentences using pre-trained embeddings create a gap due to different optimization objectives. |
| Approach: | They propose a pre-trained embedding process that optimizes informative sentences . they use sentence-word bipartite graphs to model intra-sentential distinctive features . |
| Outcome: | The proposed model outperforms heavy BERT- or RoBERTa-based sentence ranking methods by providing summary-worthy representations. |