How Large Language Models Balance Internal Knowledge with User and Document Assertions (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models often need to balance their internal parametric knowledge with external information, such as user beliefs and content from retrieved documents, in real-world scenarios like RAG or chat-based systems. |
| Approach: | They propose a three-source interaction framework to evaluate 27 large language models from 3 families on 2 datasets. |
| Outcome: | The proposed framework systematically evaluates 27 large language models from 3 families on 2 datasets. |
Similar Papers
Knowledge Conflicts for LLMs: A Survey (2024.emnlp-main)
Copied to clipboard
| Challenge: | This survey examines knowledge conflicts for large language models (LLMs) this survey aims to shed light on strategies for improving the robustness of LLMs . |
| Approach: | They focus on three categories of knowledge conflicts: context-memory, inter-context, and intra-membry conflict. |
| Outcome: | The findings highlight the challenges faced by large language models when blending contextual and parametric knowledge. |
Task Matters: Knowledge Requirements Shape LLM Responses to Context–Memory Conflict (2026.findings-acl)
Copied to clipboard
| Challenge: | Prior work has shown that large language models favor parametric knowledge under conflict, but this setting assumes that tasks should always rely on the provided passage. |
| Approach: | They propose a model-agnostic diagnostic framework that holds underlying knowledge constant while injecting controlled conflicts across tasks with varying knowledge requirements. |
| Outcome: | Evaluating representative open-source LLMs, the proposed framework holds underlying knowledge constant while injecting controlled conflicts across tasks with varying knowledge requirements. |
I Know, but I Don’t Know! How Persona Conflict Undermines Instruction Adherence in Large Language Models (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies on persona-grounded dialogue assume idealized scenarios where persona and user utterances are fully aligned. |
| Approach: | They propose a taxonomy that categorizes model behaviors into three response types . they propose sycophantic, adherent, and wavering responses as response types. |
| Outcome: | The proposed framework categorizes model behaviors into three response types and develops a measurement schema grounded in this taxonomy. |
Bias in the Mirror : Are LLMs opinions robust to their own adversarial attacks (2025.acl-long)
Copied to clipboard
| Challenge: | Existing work on large language models lacks robustness, highlighting the limitations of such models. |
| Approach: | They propose a novel approach where two LLMs engage in self-debate to persuade a neutral version of the model. |
| Outcome: | The proposed approach examines whether large language models are robust during interactions and whether they are susceptible to reinforcing misinformation or shifting to harmful viewpoints. |
Whose Facts Win? LLM Source Preferences under Knowledge Conflicts (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies on the role of the source of knowledge conflicts have not investigated the role . |
| Approach: | They propose a framework that reduces repetition bias by up to 79.2% while maintaining at least 72.5% of original preferences. |
| Outcome: | The proposed method reduces repetition bias by up to 79.2% while maintaining at least 72.5% of original preferences. |
Can Multiple Responses from an LLM Reveal the Sources of Its Uncertainty? (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models can produce unreliable or misleading outputs, posing challenges for real-world applications. |
| Approach: | They employ an auxiliary LLM to analyze the patterns of disagreement among LLMs . they validate their framework on AmbigQA, OpenBookQA, and MMLU-Pro . |
| Outcome: | The proposed model can be used to diagnose uncertainty sources in a model with an auxiliary model. |
Assessing and Mitigating Medical Knowledge Drift and Conflicts in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Rapid medical concept drift can lead LLMs to provide incorrect or outdated advice. |
| Approach: | They propose to evaluate how large language models manage knowledge conflicts in clinical guidelines. |
| Outcome: | The proposed benchmark evaluates how LLMs manage varied knowledge conflicts in clinical guidelines. |
Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidence (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on question answering models relies on retrieved documents for provenance, but recent studies show that models can retain vast amounts of factual knowledge . retrieval-based generation approaches combine parametric knowledge sources with a large number of retrieved evidence documents, achieving state-of-the-art performance on open retrieval datasets. |
| Approach: | They propose to use parametric and parametric knowledge to generate free-form questions from retrieved evidence documents. |
| Outcome: | The proposed model can use parametric and parametric knowledge to generate free-form answers from retrieved evidence documents. |
The Model Agreed, But Didn’t Learn: Diagnosing Surface Compliance in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models internalize vast world knowledge as parametric memory, yet inherit the staleness and errors of their source corpora. |
| Approach: | They propose a framework that subjects models to discriminative self-assessment under diverse contextual pressures to scrutinize subtle behavioral nuances induced by memory modifications. |
| Outcome: | The proposed framework achieves high benchmarks without overwriting internal beliefs, while recursive modifications accumulate representational residues, triggering cognitive instability and permanently diminishing the reversibility of the model’s memory state. |
Tracking the Limits of Knowledge Propagation: How LLMs Fail at Multi-Step Reasoning with Conflicting Knowledge (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for analyzing the performance of Large Language Models (LLMs) focus on single knowledge updates and fact recall, but do not consider how these updates affect downstream reasoning. |
| Approach: | They propose a benchmark to study how LLMs propagate new knowledge when it conflicts with the model's parametric knowledge. |
| Outcome: | The proposed benchmark compared models with no updated facts to show that the new methods worsen performance and improve reasoning performance. |