Mitigating Hallucination in Abstractive Summarization with Domain-Conditional Mutual Information (2024.findings-naacl)
Copied to clipboard
| Challenge: | Prior studies have attempted to enhance faithfulness of abstractive summarization, yet hallucination remains a persistent challenge. |
| Approach: | They propose a decoding strategy that adjusts the generation probability of each token by comparing it with the token’s marginal probability within the domain of the source text. |
| Outcome: | The proposed method significantly improves faithfulness and source relevance on the XSUM dataset. |
Similar Papers
Mutual Information Alleviates Hallucinations in Abstractive Summarization (2022.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive summarization models exhibit the tendency to hallucinate, i.e., output content not supported by the source document. |
| Approach: | They propose a decoding strategy that optimizes for pointwise mutual information of source and target tokens when models exhibit uncertainty. |
| Outcome: | The proposed method decreases the probability of hallucinated tokens while maintaining the Rouge and BERT-S scores of top-performing decoding strategies. |
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization (2022.acl-long)
Copied to clipboard
| Challenge: | State-of-the-art abstractive summarization systems often generate hallucinations, i.e., content that is not directly inferable from the source document. |
| Approach: | They propose a detection approach that separates factual from non-factual hallucinations of entities by masked language models. |
| Outcome: | The proposed method outperforms baselines in accuracy and F1 scores and has a strong correlation with human judgments on factuality classification tasks. |
Reducing Quantity Hallucinations in Abstractive Summarization (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Abstractive summaries are subject to hallucination, but they are not very informative. |
| Approach: | They propose to use a beam-worth of abstractive summaries to up-rank summary that is not supported by the original text. |
| Outcome: | The proposed system up-ranks summaries whose quantity terms are supported by the original text without losing Recall, and shows higher Precision. |
From Single to Multi: How LLMs Hallucinate in Multi-Document Summarization (2025.findings-naacl)
Copied to clipboard
| Challenge: | a recent study investigated hallucinations in multi-document summarization tasks . but, it is unclear how challenges arising from handling multiple documents affect outputs . |
| Approach: | They investigate how hallucinations manifest in large language models when summarizing topic-specific information from a set of documents. |
| Outcome: | The proposed benchmarks show that the models generate more hallucinations than baselines . the results highlight the need for more effective approaches to mitigate hallucinosity in MDS . |
ACUEval: Fine-grained Hallucination Evaluation and Correction for Abstractive Summarization (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent-proposed evaluation metrics for large language models have a preference-bias . however, such metrics often lack interpretability and only offer a single score . |
| Approach: | They propose a metric that leverages the power of large language models to perform two sub-tasks: decomposing summaries into atomic content units and validating them against the source document. |
| Outcome: | The proposed metric improves faithfulness scores on three summarization evaluation benchmarks by 3% compared to the next-best metric. |
Faithful to the Document or to the World? Mitigating Hallucinations via Entity-Linked Knowledge in Abstractive Summarization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing abstractive summarization systems are hampered by content hallucinations in which models generate text that is not directly inferable from the source alone. |
| Approach: | They propose to use external knowledge to latently connect entities and concepts to latences to lend provenance to many of these unfaithful yet factual entities. |
| Outcome: | The proposed model can be used to improve the factuality of summarizations without simply making them more extractive. |
On Faithfulness and Factuality in Abstractive Summarization (2020.acl-main)
Copied to clipboard
| Challenge: | Existing conditional text generation models produce unfaithful and unfaithed summaries . current models accomplish a high level of fluency and coherence . |
| Approach: | They propose to use pretrained models for document summarization to better understand hallucinations . they find that textual entailment measures better correlate with faithfulness . |
| Outcome: | The proposed models generate faithful and factual summaries as evaluated by humans. |
Mitigating Intrinsic Named Entity-Related Hallucinations of Abstractive Text Summarization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Abstractive text summarization (ATS) is important and challenging, but some hallucinations remain a challenge. |
| Approach: | They propose an adaptive margin ranking loss to facilitate two entity-alignment learning methods to tackle named entity-related hallucinations. |
| Outcome: | The proposed method improves the baseline model on automatic evaluation scores. |
Hallucination Diversity-Aware Active Learning for Text Summarization (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for alleviating hallucinations require costly human annotations . Existing approaches focus on a specific type of hallucinism, which limits their effectiveness . |
| Approach: | They propose a method to detect hallucinations from errors in semantic frame, discourse and content verifiability in LLM summarization using HAllucination Diversity-Aware Sampling. |
| Outcome: | The proposed framework reduces the need for costly human annotations to correct hallucinations in LLM outputs. |
Learning with Rejection for Abstractive Text Summarization (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing abstractive summarization systems produce non-factual summaries due to noise in the training dataset. |
| Approach: | They propose a training objective for abstractive summarization based on rejection learning that learns whether or not to reject potentially noisy tokens. |
| Outcome: | The proposed method significantly improves the factuality of generated summaries in automatic and human evaluations when compared to baseline models. |