ART: Attention Replacement Technique to Improve Factuality in LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to mitigate hallucinations in large language models are expensive and require significant resources. |
| Approach: | They propose a training-free method that replaces uniform attention patterns in shallow layers with local attention patterns to reduce hallucinations. |
| Outcome: | The proposed method reduces hallucinations across multiple LLM architectures. |
Similar Papers
Understanding New-Knowledge-Induced Factual Hallucinations in LLMs: Analysis and Interpretation (2026.findings-acl)
Copied to clipboard
| Challenge: | Prior studies have shown that fine-tuning on new knowledge can induce factual hallucinations in large language models (LLMs), leading to incorrect outputs when evaluated on previously known information. |
| Approach: | They propose to conduct a fine-grained analysis of large language models using a dataset Biography-Reasoning and QA and knowledge reasoning tasks to understand their findings. |
| Outcome: | The proposed model is able to perform a range of downstream tasks without requiring a large amount of knowledge and is compared with a control dataset. |
Zero-Resource Hallucination Prevention for Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for detecting hallucinations post-generation suffer from inconsistent performance due to the influence of instruction format and model style. |
| Approach: | They propose a new technique that evaluates the model’s familiarity with the concepts present in the input instruction and withholding the generation of response in case of unfamiliar concepts under the zero-resource setting. |
| Outcome: | The proposed technique shows superior performance across four different large language models and demonstrates that it can be used to mitigate hallucinations in LLMs. |
Shallow Focus, Deep Fixes: Enhancing Shallow Layers Vision Attention Sinks to Alleviate Hallucination in LVLMs (2025.emnlp-main)
Copied to clipboard
Xiaofeng Zhang, Yihao Quan, Chen Shen, Chaochen Gu, Xiaosong Yuan, Shaotian Yan, Jiawei Cao, Hao Cheng, Kaijie Wu, Jieping Ye
| Challenge: | Multimodal large language models (MLLMs) demonstrate excellent abilities for understanding visual information, but the hallucination remains a challenging problem. |
| Approach: | They propose a training-free approach to enhance vision attention sinks to facilitate convergence of the image token attention sink within shallow layers. |
| Outcome: | The proposed approach improves the convergence of the image token attention sink within shallow layers and strengthens the layer’s focus on the image itself. |
Do Robot Snakes Dream like Electric Sheep? Investigating the Effects of Architectural Inductive Biases on Hallucination (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have a tendency to hallucinate false or misleading information, limiting their reliability. |
| Approach: | They examine how architecture-based inductive biases affect the propensity to hallucinate . they find that the models are more reliable and more reliable than traditional models . |
| Outcome: | The proposed models can be used to train and train large language models that are factual or able to explain themselves through their knowledge. |
The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | a growing number of researchers are studying the hallucination issue in large language models. |
| Approach: | They propose a hallucination detection benchmark and a method to detect hallucines in LLMs. |
| Outcome: | The proposed method detects hallucinations and mitigates them using different training stages. |
Tutorial Proposal: Hallucination in Large Language Models (2024.lrec-tutorials)
Copied to clipboard
| Challenge: | Grasping the intricacies of hallucination in LLMs can be daunting, especially for those new to the field. |
| Approach: | This tutorial aims to bridge the gap between the field and the field of hallucination . it will explore the key aspects of hallucinonation, including benchmarking, detection, and mitigation techniques . |
| Outcome: | This tutorial will explore the key aspects of hallucination in LLMs . it will also explore the specific constraints and shortcomings of current approaches . |
Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps (2024.emnlp-main)
Copied to clipboard
| Challenge: | Despite the utility and impressive capabilities of large language models, their tendency to generate hallucinations presents a significant challenge in their deployment. |
| Approach: | They propose a simple hallucination detection model based on the ratio of attention weights on the context versus newly generated tokens. |
| Outcome: | The proposed model reduces the amount of hallucinations by 9.6% in a summarization task. |
DAMRO: Dive into the Attention Mechanism of LVLM to Reduce Object Hallucination (2024.emnlp-main)
Copied to clipboard
| Challenge: | Despite the success of Large Vision-Language Models, they suffer from hallucination. |
| Approach: | They propose a training-free strategy that "D**ive into" the attention of LVLMs to "R**educe" object hallucination by using classification tokens of ViT. |
| Outcome: | The proposed method reduces the impact of outlier tokens on LVLMs . the proposed method is based on LLaVA-1.5, LLvaVA-NeXT and InstructBLIP . |
Whispers that Shake Foundations: Analyzing and Mitigating False Premise Hallucinations in Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models generate hallucinated text when confronted with false premise questions . authors propose a method to mitigate false premises hallucinosity . |
| Approach: | They propose a method to constrain false premise attention heads during the model inference process. |
| Outcome: | The proposed method improves performance by constraining false premise attention heads . it yields a notable increase of nearly 20% of model performance . |
Addressing Bias and Hallucination in Large Language Models (2024.lrec-tutorials)
Copied to clipboard
Nihar Ranjan Sahoo, Ashita Saxena, Kishan Maharaj, Arif A. Ahmad, Abhijit Mishra, Pushpak Bhattacharyya
| Challenge: | This tutorial provides a comprehensive overview of two critical aspects of Large Language Models: bias and hallucination. |
| Approach: | This tutorial provides an overview of two critical aspects of Large Language Models: bias and hallucination. |
| Outcome: | This tutorial delves into the complex dimensions of Large Language Models (LLMs) it outlines ethical considerations pertinent to their development and discusses hallucination, a prevalent issue in generative AI systems such as LLMs. |