Faithful to the Document or to the World? Mitigating Hallucinations via Entity-Linked Knowledge in Abstractive Summarization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing abstractive summarization systems are hampered by content hallucinations in which models generate text that is not directly inferable from the source alone. |
| Approach: | They propose to use external knowledge to latently connect entities and concepts to latences to lend provenance to many of these unfaithful yet factual entities. |
| Outcome: | The proposed model can be used to improve the factuality of summarizations without simply making them more extractive. |
Similar Papers
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization (2022.acl-long)
Copied to clipboard
| Challenge: | State-of-the-art abstractive summarization systems often generate hallucinations, i.e., content that is not directly inferable from the source document. |
| Approach: | They propose a detection approach that separates factual from non-factual hallucinations of entities by masked language models. |
| Outcome: | The proposed method outperforms baselines in accuracy and F1 scores and has a strong correlation with human judgments on factuality classification tasks. |
On Faithfulness and Factuality in Abstractive Summarization (2020.acl-main)
Copied to clipboard
| Challenge: | Existing conditional text generation models produce unfaithful and unfaithed summaries . current models accomplish a high level of fluency and coherence . |
| Approach: | They propose to use pretrained models for document summarization to better understand hallucinations . they find that textual entailment measures better correlate with faithfulness . |
| Outcome: | The proposed models generate faithful and factual summaries as evaluated by humans. |
Entity-level Factual Consistency of Abstractive Text Summarization (2021.eacl-main)
Copied to clipboard
Feng Nan, Ramesh Nallapati, Zhiguo Wang, Cicero Nogueira dos Santos, Henghui Zhu, Dejiao Zhang, Kathleen McKeown, Bing Xiang
| Challenge: | Existing models exhibit entity hallucination, generating names of entities that are not present in the source document. |
| Approach: | They propose to use entity-level factual consistency to improve model quality . they propose to filter the training data to reduce entity hallucination problem . |
| Outcome: | The proposed model can reduce the entity hallucination problem by filtering the training data. |
Not all Hallucinations are Good to Throw Away When it Comes to Legal Abstractive Summarization (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing models for summarization of legal documents rely on external knowledge to generate abstracts. |
| Approach: | They propose an entity-driven approach that learns the model to generate factual hallucinations . they evaluate legal documents in English and French to evaluate their results . |
| Outcome: | The proposed approach reduces non-factual hallucinations and maximizes summary coverage and factual hallucines at entity-level. |
Reducing Quantity Hallucinations in Abstractive Summarization (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Abstractive summaries are subject to hallucination, but they are not very informative. |
| Approach: | They propose to use a beam-worth of abstractive summaries to up-rank summary that is not supported by the original text. |
| Outcome: | The proposed system up-ranks summaries whose quantity terms are supported by the original text without losing Recall, and shows higher Precision. |
Mitigating Intrinsic Named Entity-Related Hallucinations of Abstractive Text Summarization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Abstractive text summarization (ATS) is important and challenging, but some hallucinations remain a challenge. |
| Approach: | They propose an adaptive margin ranking loss to facilitate two entity-alignment learning methods to tackle named entity-related hallucinations. |
| Outcome: | The proposed method improves the baseline model on automatic evaluation scores. |
Improving Faithfulness by Augmenting Negative Summaries from Fake Documents (2022.emnlp-main)
Copied to clipboard
| Challenge: | Current abstractive summarization systems tend to hallucinate unfaithful content . however, the most common method does not disentangle factual errors from other errors. |
| Approach: | They propose a back-translation-style approach to augment negative samples that mimic factual errors made by the model. |
| Outcome: | The proposed method improves faithfulness without sacrificing informativeness . it incorporates negative samples into training, and produces faithful/unfaithful summaries . |
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control (2022.findings-naacl)
Copied to clipboard
| Challenge: | Abstractive summarization systems have been shown to be more prone to unfaithful facts . 30% of summaries generated by pre-trained language models suffer from hallucination . |
| Approach: | They propose a method to remedy entity-level extrinsic hallucinations with Entity Coverage Control . they first compute entity coverage precision and prepend the corresponding control code . a further fine-tuning is performed to unlock zero-shot summarization . |
| Outcome: | The proposed method leads to more faithful and salient abstractive summarization in fine-tuning and zero-shot settings. |
Mitigating Hallucination in Abstractive Summarization with Domain-Conditional Mutual Information (2024.findings-naacl)
Copied to clipboard
| Challenge: | Prior studies have attempted to enhance faithfulness of abstractive summarization, yet hallucination remains a persistent challenge. |
| Approach: | They propose a decoding strategy that adjusts the generation probability of each token by comparing it with the token’s marginal probability within the domain of the source text. |
| Outcome: | The proposed method significantly improves faithfulness and source relevance on the XSUM dataset. |
Extractive is not Faithful: An Investigation of Broad Unfaithfulness Problems in Extractive Summarization (2023.acl-long)
Copied to clipboard
| Challenge: | Abstractive summarization is less prone to unfaithfulness issues than abstractive summaries . but, unfaitfulness problems, i.e., hallucinating new information, are still a problem in extractive summarisation . |
| Approach: | They propose a typology with five types of broad unfaithfulness problems that can appear in extractive summaries, including and beyond not-entailment. |
| Outcome: | The proposed metric shows that it detects unfaithful summaries faster than existing faithfulness evaluation metrics. |