The Law of Knowledge Overshadowing: Towards Understanding, Predicting and Preventing LLM Hallucination

Challenge:	Hallucination is a persistent challenge in large language models where even with rigorous quality control, models often generate distorted facts.
Approach:	They propose a new framework to quantify factual hallucinations by modeling knowledge overshadowing.
Outcome:	The proposed framework improves model factuality on Overshadow (27.9%), MemoTrap (13.1%) and NQ-Swap (18.3%).

Similar Papers

Overview

Challenge

Approach

Outcome

Pierce the Mists, Greet the Sky: Decipher Knowledge Overshadowing via Knowledge Circuit Analysis (2025.emnlp-main)

Copied to clipboard

Haoming Huang, Yibo Yan, Jiahao Huo, Xin Zou, Xinfeng Li, Kun Wang, Xuming Hu

Reasoning

Challenge:	Large Language Models (LLMs) are hampered by hallucinations, a particularly challenging variant, knowledge overshadowing, which can lead to erroneous outputs even with high-quality training data.
Approach:	They propose a framework to analyze and detect knowledge overshadowing by using knowledge circuit analysis to dissect the function of key components in the circuit and how attention pattern dynamics contribute to the phenomenon.
Outcome:	Extensive experiments show that the framework can detect and analyze knowledge overshadowing and improves on existing models.

Understanding New-Knowledge-Induced Factual Hallucinations in LLMs: Analysis and Interpretation (2026.findings-acl)

Copied to clipboard

Renfei Dang, Peng Hu, Zhejian Lai, Changjiang Gao, Min Zhang, Shujian Huang

Reasoning

Challenge:	Prior studies have shown that fine-tuning on new knowledge can induce factual hallucinations in large language models (LLMs), leading to incorrect outputs when evaluated on previously known information.
Approach:	They propose to conduct a fine-grained analysis of large language models using a dataset Biography-Reasoning and QA and knowledge reasoning tasks to understand their findings.
Outcome:	The proposed model is able to perform a range of downstream tasks without requiring a large amount of knowledge and is compared with a control dataset.

On Large Language Models’ Hallucination with Regard to Known Facts (2024.naacl-long)

Copied to clipboard

Che Jiang, Biqing Qi, Xiangyu Hong, Dayuan Fu, Yang Cheng, Fandong Meng, Mo Yu, Bowen Zhou, Jie Zhou

Language Models

Challenge:	Large language models are successful in answering factoid questions but are also prone to hallucination.
Approach:	They propose self-reporting to the model when faced with such limitations.
Outcome:	The proposed classifier can detect hallucinations with an 88% success rate and can be used to answer factoid questions with correct answer knowledge.

The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models (2024.acl-long)

Copied to clipboard

Junyi Li, Jie Chen, Ruiyang Ren, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, Ji-Rong Wen

Language Models

Challenge:	a growing number of researchers are studying the hallucination issue in large language models.
Approach:	They propose a hallucination detection benchmark and a method to detect hallucines in LLMs.
Outcome:	The proposed method detects hallucinations and mitigates them using different training stages.

Why LLMs Hallucinate on Structured Knowledge: A Mechanistic Analysis of Reasoning over Linearized Representations (2026.acl-long)

Copied to clipboard

Shanghao Li, Jinda Han, Yibo Wang, Yuanjie Zhu, Zihe Song, Langzhou He, Kenan Kamel A Alghythee, Philip S. Yu

Semantic Text Processing

Challenge:	Existing literature primarily addresses this problem through external interventions such as retrieval augmentation and prompt engineering at the input or output level.
Approach:	They find that LLMs can still produce hallucinated outputs when using structured external knowledge.
Outcome:	The proposed models fail to ground the provided knowledge, causing the model to revert to parametric memory.

Mitigating Hallucination by Integrating Knowledge Graphs into LLM Inference – a Systematic Literature Review (2025.acl-srw)

Copied to clipboard

Robin Wagner, Emanuel Kitzelmann, Ingo Boersch

Structured Data in NLP

Challenge:	Large Language Models (LLMs) have made significant progress on different language tasks, but they tend to "hallucinate" plausible but factually incorrect answers.
Approach:	They propose to integrate knowledge graphs (KGs) into LLM inference to reduce hallucinations by searching online and applying a selection process.
Outcome:	The proposed integration improves performance on benchmark datasets and also to mitigate hallucinations.

Alleviating Hallucinations of Large Language Models through Induced Hallucinations (2025.findings-naacl)

Copied to clipboard

Yue Zhang, Leyang Cui, V. W., Shuming Shi

Reasoning

Challenge:	Existing studies have shown that large language models generate inaccurate or fabricated information, a phenomenon known as hallucinations.
Approach:	They propose a simple strategy to induce-then-contrast decode LLMs to enhance their factuality . they first induce hallucinations from the original model and penalize them .
Outcome:	The proposed strategy improves factuality of large language models across task formats, model sizes, and model families.

Addressing Bias and Hallucination in Large Language Models (2024.lrec-tutorials)

Copied to clipboard

Nihar Ranjan Sahoo, Ashita Saxena, Kishan Maharaj, Arif A. Ahmad, Abhijit Mishra, Pushpak Bhattacharyya

Responsible & Trustworthy NLP

Challenge:	This tutorial provides a comprehensive overview of two critical aspects of Large Language Models: bias and hallucination.
Approach:	This tutorial provides an overview of two critical aspects of Large Language Models: bias and hallucination.
Outcome:	This tutorial delves into the complex dimensions of Large Language Models (LLMs) it outlines ethical considerations pertinent to their development and discusses hallucination, a prevalent issue in generative AI systems such as LLMs.

The Troubling Emergence of Hallucination in Large Language Models - An Extensive Definition, Quantification, and Prescriptive Remediations (2023.emnlp-main)

Copied to clipboard

Vipula Rawte, Swagata Chakraborty, Agnibh Pathak, Anubhav Sarkar, S.M Towhidul Islam Tonmoy, Aman Chadha, Amit Sheth, Amitava Das

Language Models

Challenge:	Recent advances in Large Language Models have generated widespread acclaim, but hallucination has also emerged as a by-product.
Approach:	They propose a fine-grained discourse on profiling hallucination based on its degree, orientation, and category . they categorize hallucines into six types: acronym ambiguity, generated golem, virtual voice, geographic erratum, time wrap .
Outcome:	The proposed method categorizes hallucination into six types based on their degree, orientation, and category .

Tutorial Proposal: Hallucination in Large Language Models (2024.lrec-tutorials)

Copied to clipboard

Vipula Rawte, Aman Chadha, Amit Sheth, Amitava Das

Language Models

Challenge:	Grasping the intricacies of hallucination in LLMs can be daunting, especially for those new to the field.
Approach:	This tutorial aims to bridge the gap between the field and the field of hallucination . it will explore the key aspects of hallucinonation, including benchmarking, detection, and mitigation techniques .
Outcome:	This tutorial will explore the key aspects of hallucination in LLMs . it will also explore the specific constraints and shortcomings of current approaches .