Jointly Learning Guidance Induction and Faithful Summary Generation via Conditional Variational Autoencoders (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for abstractive summarization generate factual consistency summaries with a high level of accuracy and coherence. |
| Approach: | They propose a framework that induces the guidance information and generates summary equipment with the guidance synchronously. |
| Outcome: | The proposed framework generates fluent summaries with no constraint on the words and phrases, and is more faithful than the existing state-of-the-art approaches. |
Similar Papers
GSum: A General Framework for Guided Neural Abstractive Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Abstractive summarization models are flexible, but they can be difficult to control. |
| Approach: | They propose a general and extensible guided summarization framework that takes different kinds of guidance as input and perform experiments across different varieties. |
| Outcome: | The proposed framework can generate more faithful summaries and different types of guidance generate qualitatively different summary. |
Gradient-Based Adversarial Factual Consistency Evaluation for Abstractive Summarization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive summarization models often produce inconsistent statements or false facts. |
| Approach: | They propose an efficient weak-supervised adversarial data augmentation approach to generate factual consistency datasets by backpropagating gradients on token embeddings. |
| Outcome: | The proposed model can make interpretable factual errors tracing on public datasets and is cost-effective. |
Guided Neural Language Generation for Abstractive Summarization using Abstract Meaning Representation (D18-1)
Copied to clipboard
| Challenge: | Recent work on abstractive summarization has made progress with neural encoder-decoder architectures, but these models lack explicit semantic modeling of the source document and its summary. |
| Approach: | They extend previous work on abstractive summarization using Abstract Meaning Representation (AMR) with a neural language generation stage which they guide using the source document. |
| Outcome: | The proposed approach improves summarization performance by 7.4 and 10.5 points in ROUGE-2 using gold standard AMR parses and parses obtained from an off-the-shelf parser respectively. |
Masked Summarization to Generate Factually Inconsistent Summaries for Improved Factual Consistency Checking (2022.findings-naacl)
Copied to clipboard
| Challenge: | Abstractive summarization systems generate paraphrases, but they often contain information inconsistent with the source text. |
| Approach: | They propose to generate factually inconsistent summaries using source texts and reference summary with key information masked to train a factual consistency classifier. |
| Outcome: | The proposed method outperforms existing models and shows a competitive correlation with human judgments. |
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for generating abstractive summarization are inconsistent and rely on heuristically created data for error handling. |
| Approach: | They propose a contrastive learning formulation that leverages both positive and negative summaries to train summarization systems that are better at distinguishing between them. |
| Outcome: | The proposed learning framework produces more factual summaries than strong comparisons with post error correction, entailment-based reranking, and unlikelihood training. |
Discourse Understanding and Factual Consistency in Abstractive Summarization (2021.eacl-main)
Copied to clipboard
Saadia Gabriel, Antoine Bosselut, Jeff Da, Ari Holtzman, Jan Buys, Kyle Lo, Asli Celikyilmaz, Yejin Choi
| Challenge: | Existing abstractive summarization models often hallucinate information or generate factually incorrect summaries. |
| Approach: | They propose a general framework for abstractive summarization with factual consistency and distinct modeling of the narrative flow in an output summary. |
| Outcome: | The proposed framework generates abstracts with factual consistency and coherence significantly better than baselines. |
Evaluating the Factual Consistency of Abstractive Text Summarization (2020.emnlp-main)
Copied to clipboard
| Challenge: | a weakly-supervised approach is needed to verify factual consistency . auxiliary span extraction tasks are useful for verifying factual consistent summaries . |
| Approach: | They propose a weakly-supervised approach for verifying factual consistency . they transfer the model to summaries generated by several neural models . |
| Outcome: | The proposed approach outperforms models trained with strong supervision on source documents and human evaluations. |
Multilingual Summarization with Factual Consistency Evaluation (2023.findings-acl)
Copied to clipboard
| Challenge: | Abstractive summarization models generate factually inconsistent summaries, reducing their utility for real-world applications. |
| Approach: | They propose to use data filtering and controlled generation to detect hallucinations in machine generated summaries. |
| Outcome: | The proposed models detect factual inconsistencies in machine generated summaries, but they focus on English only. |
SummaCoz: A Dataset for Improving the Interpretability of Factual Consistency Detection for Summarization (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Summarization is an important application of Large Language Models. |
| Approach: | They integrate human-annotated and model-generated natural language explanations to elucidate how a summary deviates and becomes inconsistent with its source article. |
| Outcome: | The proposed model provides rationales for its judgments and improves its accuracy significantly. |
Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedback (2023.acl-long)
Copied to clipboard
Paul Roit, Johan Ferret, Lior Shani, Roee Aharoni, Geoffrey Cideron, Robert Dadashi, Matthieu Geist, Sertan Girgin, Leonard Hussenot, Orgad Keller, Nikola Momchev, Sabela Ramos Garea, Piotr Stanczyk, Nino Vieillard, Olivier Bachem, Gal Elidan, Avinatan Hassidim, Olivier Pietquin, Idan Szpektor
| Challenge: | Recent advances in abstractive summarization systems produce factually inconsistent text . this is emphasized in tasks like summarizing, which often produce inconsistent text with no input article . |
| Approach: | They use reinforcement learning to optimize for factual consistency and explore trade-offs . they use textual-entailment rewards to optimize the accuracy of the generated summaries . |
| Outcome: | The proposed method improves faithfulness, salience and conciseness of the generated summaries. |