Understanding New-Knowledge-Induced Factual Hallucinations in LLMs: Analysis and Interpretation (2026.findings-acl)
Copied to clipboard
| Challenge: | Prior studies have shown that fine-tuning on new knowledge can induce factual hallucinations in large language models (LLMs), leading to incorrect outputs when evaluated on previously known information. |
| Approach: | They propose to conduct a fine-grained analysis of large language models using a dataset Biography-Reasoning and QA and knowledge reasoning tasks to understand their findings. |
| Outcome: | The proposed model is able to perform a range of downstream tasks without requiring a large amount of knowledge and is compared with a control dataset. |
Similar Papers
Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Pre-training Large Language Models (LLMs) on textual corpora embeds substantial factual knowledge in their parameters, which is essential for excelling in various downstream applications. |
| Approach: | They propose to use supervised fine-tuning to align large language models to new factual information that is not acquired through pre-training. |
| Outcome: | The proposed model is trained to generate facts that are not grounded in pre-existing knowledge, but hallucinates when examples with new knowledge are learned. |
On Large Language Models’ Hallucination with Regard to Known Facts (2024.naacl-long)
Copied to clipboard
Che Jiang, Biqing Qi, Xiangyu Hong, Dayuan Fu, Yang Cheng, Fandong Meng, Mo Yu, Bowen Zhou, Jie Zhou
| Challenge: | Large language models are successful in answering factoid questions but are also prone to hallucination. |
| Approach: | They propose self-reporting to the model when faced with such limitations. |
| Outcome: | The proposed classifier can detect hallucinations with an 88% success rate and can be used to answer factoid questions with correct answer knowledge. |
Unfamiliar Finetuning Examples Control How Language Models Hallucinate (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) generate plausible-sounding responses that are factually incorrect. |
| Approach: | They propose an approach to learn more reliable reward models by modifying how unfamiliar finetuning examples are supervised to influence model responses to unfamiliar queries. |
| Outcome: | The proposed approach improves the efficacy of RL factuality finetuning in long-form biography and book/movie plot generation tasks. |
The Law of Knowledge Overshadowing: Towards Understanding, Predicting and Preventing LLM Hallucination (2025.findings-acl)
Copied to clipboard
Yuji Zhang, Sha Li, Cheng Qian, Jiateng Liu, Pengfei Yu, Chi Han, Yi R. Fung, Kathleen McKeown, ChengXiang Zhai, Manling Li, Heng Ji
| Challenge: | Hallucination is a persistent challenge in large language models where even with rigorous quality control, models often generate distorted facts. |
| Approach: | They propose a new framework to quantify factual hallucinations by modeling knowledge overshadowing. |
| Outcome: | The proposed framework improves model factuality on Overshadow (27.9%), MemoTrap (13.1%) and NQ-Swap (18.3%). |
The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | a growing number of researchers are studying the hallucination issue in large language models. |
| Approach: | They propose a hallucination detection benchmark and a method to detect hallucines in LLMs. |
| Outcome: | The proposed method detects hallucinations and mitigates them using different training stages. |
Why LLMs Hallucinate on Structured Knowledge: A Mechanistic Analysis of Reasoning over Linearized Representations (2026.acl-long)
Copied to clipboard
Shanghao Li, Jinda Han, Yibo Wang, Yuanjie Zhu, Zihe Song, Langzhou He, Kenan Kamel A Alghythee, Philip S. Yu
| Challenge: | Existing literature primarily addresses this problem through external interventions such as retrieval augmentation and prompt engineering at the input or output level. |
| Approach: | They find that LLMs can still produce hallucinated outputs when using structured external knowledge. |
| Outcome: | The proposed models fail to ground the provided knowledge, causing the model to revert to parametric memory. |
Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Survey (2024.naacl-long)
Copied to clipboard
| Challenge: | Increasing the use of knowledge graphs to augment LLMs has led to hallucinations . large language models (LLMs) are prone to producing hallucinosis due to knowledge gaps . |
| Approach: | They review knowledge graph-based augmentation techniques in large language models to assess their effectiveness and examine their performance. |
| Outcome: | The proposed methods have been evaluated against three groups of LLMs and offer methodological comparisons and performance evaluations. |
Trust Me, I’m Wrong: LLMs Hallucinate with Certainty Despite Knowing the Answer (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work on large language model (LLM) hallucinations associated with model uncertainty or inaccurate knowledge. |
| Approach: | They define and investigate a type of hallucination where a model can answer a question correctly but a perturbation causes it to produce a hallucinous response with high certainty. |
| Outcome: | The proposed mitigations outperform existing methods on CHOKE hallucinations . the findings highlight the need to understand their origins and improve mitigation strategies . |
Fine-Grained Detection of Context-Grounded Hallucinations Using LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing representations of hallucinations limit the types of errors that can be expressed, so we propose a new representation based on free-form textual descriptions, capturing the full range of possible errors. |
| Approach: | They propose a benchmark for localizing hallucinations using LLMs with a human annotation of over 1,000 examples and a protocol to verify its quality in a humans evaluation. |
| Outcome: | The proposed representation captures the full range of possible errors, and the best model achieves an F1 score of 0.67. |
Mitigating Hallucination by Integrating Knowledge Graphs into LLM Inference – a Systematic Literature Review (2025.acl-srw)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have made significant progress on different language tasks, but they tend to "hallucinate" plausible but factually incorrect answers. |
| Approach: | They propose to integrate knowledge graphs (KGs) into LLM inference to reduce hallucinations by searching online and applying a selection process. |
| Outcome: | The proposed integration improves performance on benchmark datasets and also to mitigate hallucinations. |