FaithfulRAG: Fact-Level Conflict Modeling for Context-Faithful Retrieval-Augmented Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing faithful RAG approaches enforce strict context adherence, but they forcibly suppress the model’s parametric knowledge, which undermines the model's internal knowledge structure and increases the risk of misinterpreting the context. |
| Approach: | They propose a framework that resolves knowledge conflicts by explicitly modeling discrepancies between the model’s parametric knowledge and retrieved context. |
| Outcome: | The proposed framework outperforms state-of-the-art methods in knowledge conflict cases and identifies conflicting knowledge at the fact level and designs a self-thinking process. |
Similar Papers
Beyond Black-Box Interventions: Latent Probing for Faithful Retrieval-Augmented Generation (2026.findings-acl)
Copied to clipboard
Linfeng Gao, Qinggang Zhang, Baolong Bi, Bo Zeng, Zheng Yuan, Zerui Chen, Zhimin Wei, Shenghua Liu, Linlong Xu, Longyue Wang, Weihua Luo, Jinsong Su
| Challenge: | Existing approaches to improve contextual faithfulness treat the LLM as a black box, generating responses that are inconsistent with the provided context. |
| Approach: | They propose a framework for faithful RAG that operates in three stages: (i) fine-grained knowledge pruning to filter irrelevant context, (ii) latent conflict probing to identify hard conflicts in the model’s latent space, and (iv) conflict-aware attention to modulate attention heads toward faithful context integration. |
| Outcome: | Experiments show that ProbeRAG significantly improves both accuracy and contextual faithfulness. |
Context-faithful Prompting for Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models encode parametric knowledge about world facts but overly rely on it can cause incorrect predictions in context-sensitive NLP tasks. |
| Approach: | They propose to use opinion-based prompts and counterfactual demonstrations to improve LLM faithfulness to contexts. |
| Outcome: | The proposed methods improve faithfulness to contexts using opinion-based prompts and counterfactual demonstrations. |
Context-DPO: Aligning Language Models for Context-Faithfulness (2025.findings-acl)
Copied to clipboard
Baolong Bi, Shaohan Huang, Yiwei Wang, Tianchi Yang, Zihan Zhang, Haizhen Huang, Lingrui Mei, Junfeng Fang, Zehao Li, Furu Wei, Weiwei Deng, Feng Sun, Qi Zhang, Shenghua Liu
| Challenge: | Context-DPO is the first alignment method specifically designed to enhance contextfaithfulness for large language models. |
| Approach: | They propose a benchmark that simulates Retrieval-Augmented Generation scenarios with knowledge conflicts to evaluate context-faithfulness. |
| Outcome: | The proposed method improves LLMs' context-faithfulness by 35% to 280% over open-source models. |
Guaranteeing Knowledge Integration with Joint Decoding for Retrieval-Augmented Generation (2026.acl-long)
Copied to clipboard
Zhengyi Zhao, Shubo Zhang, Zezhong Wang, Yuxi Zhang, Huimin Wang, Yutian Zhao, Yefeng Zheng, Binyang Li, Kam-Fai Wong, Xian Wu
| Challenge: | Retrieval-Augmented Generation (RAG) provides access to external knowledge, but current research focuses on retrieval quality and 'integration bottleneck' . |
| Approach: | They propose a framework that explicitly decouples reasoning from evidence integration by generating an 'Inner-Answer' and a 'Refer-Aswer" they propose 'a joint decoding mechanism that dynamically fuses the logical coherence of the Inner-Andswer with the factual precision of the Refer-Adswer at the token level' |
| Outcome: | The proposed framework improves accuracy by 12.1% and reduces hallucinations by 16.3% on five QA benchmarks. |
Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies have not linked the behavior of retrieval augmented generation (RAG) with imperfect retrieval, including irrelevant, misleading, or even malicious information. |
| Approach: | They propose an approach that integrates external knowledge with source-awareness to overcome imperfect retrieval errors in RAG. |
| Outcome: | The proposed approach is superior to previous robustness-enhanced approaches under the worst-case scenario. |
CausalRAG: Integrating Causal Graphs into Retrieval-Augmented Generation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing RAG frameworks face critical limitations due to text chunking and semantic similarity. |
| Approach: | They propose a framework that incorporates causal graphs into the retrieval process. |
| Outcome: | The proposed framework preserves contextual continuity and improves retrieval precision, leading to more accurate and interpretable responses. |
Knowledge Conflicts for LLMs: A Survey (2024.emnlp-main)
Copied to clipboard
| Challenge: | This survey examines knowledge conflicts for large language models (LLMs) this survey aims to shed light on strategies for improving the robustness of LLMs . |
| Approach: | They focus on three categories of knowledge conflicts: context-memory, inter-context, and intra-membry conflict. |
| Outcome: | The findings highlight the challenges faced by large language models when blending contextual and parametric knowledge. |
LLMs Trust Humans More, That’s a Problem! Unveiling and Mitigating the Authority Bias in Retrieval-Augmented Generation (2025.acl-long)
Copied to clipboard
Yuxuan Li, Xinwei Guo, Jiashi Gao, Guanhua Chen, Xiangyu Zhao, Jiaxin Zhang, Quanying Liu, Haiyan Wu, Xin Yao, Xuetao Wei
| Challenge: | Large language models (LLMs) generate outputs that stray from user input or contravene established knowledge. |
| Approach: | They propose a new phenomenon, Authority Bias, where LLMs favor one knowledge source over the other . they propose atomic information that generates conflicts and a Conflict Detection Enhanced Query framework . |
| Outcome: | The proposed framework reduces Authority bias in large language models . it detects conflicts, performs credibility assessment on conflicting paragraphs, and detects perturbed text . |
LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answering (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing long-context Large Language Models (LLMs) struggle with the “lost in the middle” issue. |
| Approach: | They propose a general, dual-perspective, and robust LLM-based RAG system paradigm for LCQA to enhance RAG’s understanding of complex long-context knowledge. |
| Outcome: | The proposed system outperforms long-context LLMs, advanced RAG, and vanilla RAG on three multi-hop datasets. |
Conflict-Aware Soft Prompting for Retrieval-Augmented Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that REtrieval-augmented generation (RAG) fails to resolve the conflict between incorrect external context and correct parametric knowledge. |
| Approach: | They propose a conflict-aware REtrieval-augmented generation system that encodes external context into compact memory embeddings and captures a guidance signal that directs reasoning toward the more reliable knowledge source. |
| Outcome: | Extensive experiments show that CARE effectively mitigates context-memory conflicts, leading to an average performance gain of 5.0% on QA and fact-checking benchmarks. |