Gradient-guided Attention Map Editing: Towards Efficient Contextual Hallucination Mitigation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) often experience “contextual hallucination” where they prioritize self-generated content over input context, leading to a disregard for pertinent details. |
| Approach: | They propose a method that dynamically adjusts attention maps to enhance contextual relevance by using a trained classifier to identify attention maps likely to induce hallucinations. |
| Outcome: | The proposed approach reduces hallucinations across open-source models on summarization and open-book QA tasks. |
Similar Papers
Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps (2024.emnlp-main)
Copied to clipboard
| Challenge: | Despite the utility and impressive capabilities of large language models, their tendency to generate hallucinations presents a significant challenge in their deployment. |
| Approach: | They propose a simple hallucination detection model based on the ratio of attention weights on the context versus newly generated tokens. |
| Outcome: | The proposed model reduces the amount of hallucinations by 9.6% in a summarization task. |
Dynamic Attention-Guided Context Decoding for Mitigating Context Faithfulness Hallucinations in Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods, such as a n-terminal coding, do not provide accurate data for large language models. |
| Approach: | They propose a lightweight framework that leverages attention distributions and uncertainty signals in a single-pass decoding. |
| Outcome: | Experiments on open-book QA datasets show that DAGCD improves faithfulness and robustness while preserving computational efficiency. |
ALPS: Attention Localization and Pruning Strategy for Efficient Adaptation of Large Language Models (2025.findings-acl)
Copied to clipboard
Hao Chen, Haoze Li, Zhiqing Xiao, Lirong Gao, Qi Zhang, Xiaomeng Hu, Ningtao Wang, Xing Fu, Junbo Zhao
| Challenge: | Prior research has focused on optimizing general-purpose large language models to downstream tasks . however, these approaches inherently introduce data dependency, which hinders generalization and reusability. |
| Approach: | They propose an algorithm that localizes the most task-sensitive attention heads and prunes by restricting attention training updates to these heads, thereby reducing alignment costs. |
| Outcome: | The proposed algorithm achieves 2% performance improvement over baselines on three tasks while localizing the most task-sensitive attention heads. |
VADE: Visual Attention Guided Hallucination Detection and Elimination (2025.findings-acl)
Copied to clipboard
| Challenge: | Vision Language Models (VLMs) are prone to hallucinations, generating outputs that lack grounding in the actual visual data. |
| Approach: | They propose a sequence modelling approach to learn complex sequential patterns from transformer attention maps. |
| Outcome: | The proposed approach achieves an average PR-AUC of 80% in hallucination detection on M-HalDetect and an 5% improvement in hallucinosis mitigation on MSCOCO. |
CoDA: Restoring Contextual Dominance via Copy-Encouraged Attention Intervention for Mitigating RAG Hallucinations (2026.findings-acl)
Copied to clipboard
| Challenge: | Retrieval-augmented generation reduces hallucination by grounding outputs in external evidence. |
| Approach: | They propose a lightweight inference-time attention intervention that amplifies evidence-aligned value states to enhance contextual faithfulness and reduce hallucination. |
| Outcome: | The proposed model reduces hallucination by grounding model outputs in external evidence. |
Revealing and Enhancing Core Visual Regions: Harnessing Internal Attention Dynamics for Hallucination Mitigation in LVLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing training-free methods are vulnerable to the attention sink phenomenon . Existing methods include contrastive decoding and auxiliary expert models . |
| Approach: | They propose a training-free attention intervention that constructs a PAD map to identify semantically core visual regions and applies per-head Median Absolute Deviation Scaling to adaptively control the intervention strength. |
| Outcome: | The proposed intervention improves visual grounding and reduces hallucinations on multiple LVLMs and benchmarks. |
Analysis of Plan-based Retrieval for Grounded Text Generation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large, parametric language models (LLMs) produce fluent text for many applications . hallucinations are generation of text that is factually correct and semantically plausible . |
| Approach: | They propose to use learning-tuned LLMs to infuse models with retrieval mechanisms to reduce hallucinations. |
| Outcome: | The proposed approach reduces the frequency of hallucinations by reducing the coverage of relevant facts and generating more informative responses while providing higher attribution rates. |
Mitigating Hallucinations in VLMs: Enhancing Visual Attention via Head-Wise Perturbation (2026.findings-acl)
Copied to clipboard
| Challenge: | Vision–Language Models (VLMs) have demonstrated strong capabilities in tasks that require joint understanding of text and images. |
| Approach: | They propose a strategy that incorporates head-wise attention perturbation via continuous multiplicative noise coupled with a visual-guided loss focused on vision-sensitive text tokens to promote a more balanced attention distribution. |
| Outcome: | The proposed approach outperforms baseline models on three benchmarks and consistently outperformed the baseline model. |
HICD: Hallucination-Inducing via Attention Dispersion for Contrastive Decoding to Mitigate Hallucinations in Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) often generate hallucinations, producing outputs that are contextually inaccurate or factually incorrect. |
| Approach: | They propose a method that selects attention heads crucial to the model's prediction as inducing heads and induces hallucinations by dispersing attention of these inducers. |
| Outcome: | The proposed method significantly improves performance on tasks requiring contextual faithfulness, reading comprehension, and question answering. |
ART: Attention Replacement Technique to Improve Factuality in LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to mitigate hallucinations in large language models are expensive and require significant resources. |
| Approach: | They propose a training-free method that replaces uniform attention patterns in shallow layers with local attention patterns to reduce hallucinations. |
| Outcome: | The proposed method reduces hallucinations across multiple LLM architectures. |