| Challenge: | Existing abstractive summarization systems generate incorrect facts with respect to the source text. |
| Approach: | They propose a suite of two factual correction models that leverages question-answering knowledge to make corrections in system-generated summaries via span selection. |
| Outcome: | The proposed model improves factuality of news summarization without sacrificing summary quality. |
Similar Papers
Evaluating the Factual Consistency of Abstractive Text Summarization (2020.emnlp-main)
Copied to clipboard
| Challenge: | a weakly-supervised approach is needed to verify factual consistency . auxiliary span extraction tasks are useful for verifying factual consistent summaries . |
| Approach: | They propose a weakly-supervised approach for verifying factual consistency . they transfer the model to summaries generated by several neural models . |
| Outcome: | The proposed approach outperforms models trained with strong supervision on source documents and human evaluations. |
Factual Error Correction for Abstractive Summarization Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for abstractive summarization are unable to ensure factual consistency of generated summaries. |
| Approach: | They propose a post-editing corrector module to identify and correct factual errors in generated summaries. |
| Outcome: | The proposed model outperforms existing models on CNN/DailyMail dataset on factual consistency evaluation. |
Gradient-Based Adversarial Factual Consistency Evaluation for Abstractive Summarization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive summarization models often produce inconsistent statements or false facts. |
| Approach: | They propose an efficient weak-supervised adversarial data augmentation approach to generate factual consistency datasets by backpropagating gradients on token embeddings. |
| Outcome: | The proposed model can make interpretable factual errors tracing on public datasets and is cost-effective. |
Enhancing Factual Consistency of Abstractive Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Abstractive summarization models often distort or fabricate facts in articles . factual inconsistency is a common problem with abstractive summaries . |
| Approach: | They propose a fact-aware summarization model FASum to extract factual relations into the summary generation process via graph attention. |
| Outcome: | The proposed model can produce abstractive summaries with higher factual consistency compared with existing systems and corrects factual errors via modifying only a few keywords. |
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling (2022.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive summarization models often generate inconsistent summaries containing factual errors or fabricated content. |
| Approach: | They propose to generate representative examples of non-factual summaries through infilling language models and train a robust fact-correction model to post-edit them to improve factual consistency. |
| Outcome: | The proposed model outperforms previous methods in correcting factual errors on two popular summarization datasets. |
Improving Factual Consistency of Abstractive Summarization via Question Answering (2021.acl-long)
Copied to clipboard
Feng Nan, Cicero Nogueira dos Santos, Henghui Zhu, Patrick Ng, Kathleen McKeown, Ramesh Nallapati, Dejiao Zhang, Zhiguo Wang, Andrew O. Arnold, Bing Xiang
| Challenge: | Recent studies show that about 30% of summaries generated by neural text summarization suffer from fact fabrication. |
| Approach: | They propose an automatic evaluation metric to measure factual consistency and a learning algorithm that maximizes the metric during model training. |
| Outcome: | The proposed method improves factual consistency and overall quality of summarization models. |
Improving Factual Consistency in Abstractive Summarization with Sentence Structure Pruning (2024.lrec-main)
Copied to clipboard
Dingxin Hu, Xuanyu Zhang, Xingyue Zhang, Yiyang Li, Dongsheng Chen, Marina Litvak, Natalia Vanetik, Qing Yang, Dongliang Xu, Yanquan Zhou, Lei Li, Yuze Li, Yingqi Zhu
| Challenge: | Abstractive summarization models suffer from factual inconsistency problem . post-editing methods focus on replacing suspicious entities, failing to modify incorrect content hidden in sentence structures. |
| Approach: | They propose to use sentence pruning operation to correct possible errors . they propose to apply sentence pruning operations to the syntactic dependency tree . |
| Outcome: | The proposed method improves factual consistency on the FRANK dataset compared with baselines . it is model-independent and can serve as the final step in ensuring factual consistentness. |
Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reports (2020.acl-main)
Copied to clipboard
| Challenge: | Existing abstractive summarization models do not guarantee factual correctness of summaries . |
| Approach: | They propose a framework where models evaluate factual correctness by fact-checking it against its reference using an information extraction module. |
| Outcome: | The proposed method significantly improves the factual correctness and overall quality of outputs over a competitive neural summarization system, producing radiology summaries that approach the quality of human-authored ones. |
Discourse Understanding and Factual Consistency in Abstractive Summarization (2021.eacl-main)
Copied to clipboard
Saadia Gabriel, Antoine Bosselut, Jeff Da, Ari Holtzman, Jan Buys, Kyle Lo, Asli Celikyilmaz, Yejin Choi
| Challenge: | Existing abstractive summarization models often hallucinate information or generate factually incorrect summaries. |
| Approach: | They propose a general framework for abstractive summarization with factual consistency and distinct modeling of the narrative flow in an output summary. |
| Outcome: | The proposed framework generates abstracts with factual consistency and coherence significantly better than baselines. |
Improving Factuality of Abstractive Summarization without Sacrificing Summary Quality (2023.acl-short)
Copied to clipboard
| Challenge: | Recent studies have shown that most abstractive summarization models are unfaithful and suffer from a wide range of hallucination. |
| Approach: | They propose a candidate summary generation and ranking technique to improve summary factuality without sacrificing quality. |
| Outcome: | The proposed method shows that the model trained using the proposed method improves on factuality and similarity-based metrics without conflicting with the model. |