RelEdit: Evaluating Conceptual Knowledge Editing in Language Models via Relational Reasoning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing knowledge editing methods struggle to reason about related conceptual knowledge effectively, despite a lack of model-level relational reasoning. |
| Approach: | They propose a benchmark to assess concept-level and instance-level relational reasoning abilities of edited models. |
| Outcome: | The proposed model obtains the best scores on the memory-based in-context editing baseline, MICE, suggesting a promising direction for model editing. |
Similar Papers
Editing Conceptual Knowledge for Large Language Models (2024.findings-emnlp)
Copied to clipboard
Xiaohan Wang, Shengyu Mao, Shumin Deng, Yunzhi Yao, Yue Shen, Lei Liang, Jinjie Gu, Huajun Chen, Ningyu Zhang
| Challenge: | Existing knowledge editing methods can modify concept-level definitions, but they can distort instantial knowledge in LLMs, leading to poor performance. |
| Approach: | They construct a benchmark dataset ConceptEdit and establish new metrics for evaluation to investigate the editing capability of LLMs. |
| Outcome: | The proposed methods can modify concept definitions but can distort instantial knowledge in LLMs, leading to poor performance. |
Editing the Mind of Giants: An In-Depth Exploration of Pitfalls of Knowledge Editing in Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Knowledge editing is a promising technique for updating factual knowledge in large language models (LLMs) but studies have identified side effects such as knowledge distortion and the deterioration of general abilities that have emerged after editing. |
| Approach: | They propose to evaluate the side effects of knowledge editing in large language models using metrics and benchmarks. |
| Outcome: | The results of the study highlight the limitations of current knowledge editing methods and outline potential research directions. |
DocMEdit: Towards Document-Level Model Editing (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing models only output short phrases or sentences, raising doubts about their practical usability. |
| Approach: | They propose a dataset focused on document-level model editing that aims to correct errors and outdated knowledge in Large language models (LLMs) they propose to use document-based model editing to improve model capabilities in real-world scenarios. |
| Outcome: | The proposed model editing task improves model capabilities in real-world scenarios and reduces the cost of retraining. |
AdaEdit: Advancing Continuous Knowledge Editing For Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge editing methods that can efficiently update knowledge in LLMs are limited due to budget constraints. |
| Approach: | They propose a method that can enhance the performance of edited LLMs in large-size continuous editing regimes. |
| Outcome: | Extensive empirical evaluations on multiple LLMs show that the proposed method outperforms existing methods without compromising the general abilities of these models. |
Knowledge Editing for Large Language Models (2024.lrec-tutorials)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are not immune to issues of factual accuracy or logically consistent. |
| Approach: | This tutorial will present cutting-edge methods and practical tools for editing Large Language Models (LLMs). |
| Outcome: | The aim of this course is to familiarize researchers with the latest advancements and emerging strategies in the realm of knowledge editing for LLMs. |
ConKE: Conceptualization-Augmented Knowledge Editing in Large Language Models for Commonsense Reasoning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing knowledge editing methods face limited knowledge coverage in existing knowledge bases, infeasibility of annotating labels for an overabundance of commonsense knowledge, and strict knowledge formats. |
| Approach: | They propose a framework that integrates conceptualization and instantiation into the KE pipeline for LLMs to enhance their commonsense reasoning capabilities. |
| Outcome: | The proposed framework diagnoses implausible commonsense knowledge within an LLM and augments the source knowledge to be edited with conceptualization for stronger generalizability. |
Editing Large Language Models: Problems, Methods, and Opportunities (2023.emnlp-main)
Copied to clipboard
Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, Ningyu Zhang
| Challenge: | Recent advances in model editing for LLMs have created challenges and opportunities for the community. |
| Approach: | They propose to alter the behavior of LLMs efficiently within a specific domain without negatively impacting performance across other inputs. |
| Outcome: | The proposed method alters behavior of LLMs efficiently within a specific domain without negatively impacting performance across other inputs. |
TAXI: Evaluating Categorical Knowledge Editing for Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Knowledge editing aims to inject new facts into language models to improve factuality, but current benchmarks fail to evaluate consistency, which is critical to ensure efficient, accurate, and generalizable edits. |
| Approach: | They manually create a new benchmark dataset specifically created to evaluate consistency in categorical knowledge edits. |
| Outcome: | The results show that the editors achieve marginal, yet non-random consistency, and their consistency far underperforms human baselines. |
Learning to Edit: Aligning LLMs with Knowledge Editing (2024.acl-long)
Copied to clipboard
Yuxin Jiang, Yufei Wang, Chuhan Wu, Wanjun Zhong, Xingshan Zeng, Jiahui Gao, Liangyou Li, Xin Jiang, Lifeng Shang, Ruiming Tang, Qun Liu, Wei Wang
| Challenge: | Existing knowledge editing techniques rely on memorizing updated knowledge, impeding LLMs from effectively combining the new knowledge with their inherent knowledge when answering questions. |
| Approach: | They propose a Learning to Edit framework that equips LLMs with the ability to apply updated knowledge to input questions through a two-phase process . |
| Outcome: | The proposed framework outperforms existing methods in knowledge editing tasks and compares it with four benchmarks and two LLM architectures. |
The Model Agreed, But Didn’t Learn: Diagnosing Surface Compliance in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models internalize vast world knowledge as parametric memory, yet inherit the staleness and errors of their source corpora. |
| Approach: | They propose a framework that subjects models to discriminative self-assessment under diverse contextual pressures to scrutinize subtle behavioral nuances induced by memory modifications. |
| Outcome: | The proposed framework achieves high benchmarks without overwriting internal beliefs, while recursive modifications accumulate representational residues, triggering cognitive instability and permanently diminishing the reversibility of the model’s memory state. |