DYNAMICQA: Tracing Internal Knowledge Conflicts in Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | LMs are useful in a variety of downstream applications from summarization to fact-checking, often relying on factual knowledge memorized during pre-training. |
| Approach: | They use two knowledge conflict measures and a novel dataset DYNAMICQA to examine the effect of intra-memory conflict on LMs' ability to accept contextual knowledge. |
| Outcome: | The proposed model can accept contextual knowledge with a higher degree of accuracy than models with fewer truth values. |
Similar Papers
Knowledge Conflicts for LLMs: A Survey (2024.emnlp-main)
Copied to clipboard
| Challenge: | This survey examines knowledge conflicts for large language models (LLMs) this survey aims to shed light on strategies for improving the robustness of LLMs . |
| Approach: | They focus on three categories of knowledge conflicts: context-memory, inter-context, and intra-membry conflict. |
| Outcome: | The findings highlight the challenges faced by large language models when blending contextual and parametric knowledge. |
Task Matters: Knowledge Requirements Shape LLM Responses to Context–Memory Conflict (2026.findings-acl)
Copied to clipboard
| Challenge: | Prior work has shown that large language models favor parametric knowledge under conflict, but this setting assumes that tasks should always rely on the provided passage. |
| Approach: | They propose a model-agnostic diagnostic framework that holds underlying knowledge constant while injecting controlled conflicts across tasks with varying knowledge requirements. |
| Outcome: | Evaluating representative open-source LLMs, the proposed framework holds underlying knowledge constant while injecting controlled conflicts across tasks with varying knowledge requirements. |
When Facts Change: Temporal Knowledge Conflict Resolution in LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models are increasingly used in retrieval-augmented generation systems to reconcile knowledge conflicts between parametric memory and contextual inputs. |
| Approach: | They propose to use mutability to resolve temporal misalignment in large language models to compare stable and recently updated facts from Wikidata to determine if mutable models can serve as a mediating signal in this process. |
| Outcome: | The proposed model can produce reasoning for facts that actually changed but rarely for stable ones, whereas smaller models rarely detect conflict, while larger models detect it but fail to act on mutability judgments. |
Tracking the Limits of Knowledge Propagation: How LLMs Fail at Multi-Step Reasoning with Conflicting Knowledge (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for analyzing the performance of Large Language Models (LLMs) focus on single knowledge updates and fact recall, but do not consider how these updates affect downstream reasoning. |
| Approach: | They propose a benchmark to study how LLMs propagate new knowledge when it conflicts with the model's parametric knowledge. |
| Outcome: | The proposed benchmark compared models with no updated facts to show that the new methods worsen performance and improve reasoning performance. |
Who’s Who: Large Language Models Meet Knowledge Conflicts in Practice (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent large-scale pretrained language models excel in tasks requiring natural language understanding, but they often "hallucinate" plausible but incorrect content due to outdated or incorrect pretraining information. |
| Approach: | They propose a public benchmark dataset to examine model’s behavior in knowledge conflict situations. |
| Outcome: | The proposed model induces conflicts by asking about a common property among entities having the same name, resulting in questions with up to 8 distinctive answers. |
Tug-of-War between Knowledge: Exploring and Resolving Knowledge Conflicts in Retrieval-Augmented Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing knowledge conflicts in RALMs can ensnare them in a tug-of-war between knowledge and evidence, limiting their practical applicability. |
| Approach: | They propose a method called Conflict-Disentangle Contrastive Decoding (CD2) to better calibrate the model’s confidence. |
| Outcome: | The proposed method can resolve knowledge conflicts in large language models with the help of conflict-disentangle contrast decoding (CD2) . |
Assessing and Mitigating Medical Knowledge Drift and Conflicts in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Rapid medical concept drift can lead LLMs to provide incorrect or outdated advice. |
| Approach: | They propose to evaluate how large language models manage knowledge conflicts in clinical guidelines. |
| Outcome: | The proposed benchmark evaluates how LLMs manage varied knowledge conflicts in clinical guidelines. |
How Large Language Models Balance Internal Knowledge with User and Document Assertions (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models often need to balance their internal parametric knowledge with external information, such as user beliefs and content from retrieved documents, in real-world scenarios like RAG or chat-based systems. |
| Approach: | They propose a three-source interaction framework to evaluate 27 large language models from 3 families on 2 datasets. |
| Outcome: | The proposed framework systematically evaluates 27 large language models from 3 families on 2 datasets. |
Discerning and Resolving Knowledge Conflicts through Adaptive Decoding with Contextual Information-Entropy Constraint (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing decoding methods for large language models (LLMs) are specialized in resolving knowledge conflicts and could inadvertently deteriorate performance in absence of conflicts. |
| Approach: | They propose an adaptive decoding method to discern whether knowledge conflicts occur and resolve them by a contextual information-entropy constraint decoding technique. |
| Outcome: | The proposed method improves the model’s faithfulness to conflicting context and maintains high performance among non-conflicting contexts. |
Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Models (2024.findings-acl)
Copied to clipboard
Zhuoran Jin, Pengfei Cao, Hongbang Yuan, Yubo Chen, Jiexin Xu, Huaijun Li, Xiaojian Jiang, Kang Liu, Jun Zhao
| Challenge: | Existing methods to expand internal memory boundaries of language models by providing external context can often conflict, leading to knowledge conflicts. |
| Approach: | They propose a method that prunes conflicting attention heads without updating model parameters. |
| Outcome: | The proposed method can flexibly control eight LMs to use internal memory or external context without updating model parameters. |