Papers with ZsRE
Erasing Without Remembering: Implicit Knowledge Forgetting in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | a new method for unlearning large language models is proposed to improve the performance of large language model models. |
| Approach: | They propose a probability perturbation-based unlearning paradigm that allows models to forget implicit knowledge in large language models with a focus on generalisation. |
| Outcome: | The proposed model improves unlearning vanilla target data while forgetting implicit knowledge. |
Knowledge Graph-Driven Memory Editing with Directional Interventions (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are hampered by inaccuracies and outdated information. |
| Approach: | They propose a framework that constructs knowledge graphs using available information to guide the direction of knowledge editing. |
| Outcome: | The proposed framework allows consistent, aligned, and stable information during large-scale editing scenarios. |
Model Editing by Standard Fine-Tuning (2024.findings-acl)
Copied to clipboard
| Challenge: | specialized methods for model editing are not as effective due to poor performance . standard fine-tuning alone can yield competitive model editing performance if it is modified . |
| Approach: | They propose to optimize conditional likelihood rather than the full likelihood . they also train on random or similar unedited facts to encourage locality . |
| Outcome: | The proposed model editing method outperforms specialized models in terms of edit score. |
Keys to Robust Edits: From Theoretical Insights to Practical Advances (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for modifying parametric memory are prone to inaccuracies due to conflicting or outdated information. |
| Approach: | They propose a plug-and-play module that disentangles editing keys from native model representations and dynamically adjusts keys via contrastive learning to achieve robustness-specificity balance. |
| Outcome: | The proposed method improves over robustness tests by up to 66.4% while maintaining the success rate unaffected. |
Relaxing the Constraints: A Dual-Importance Projection Mechanism for Lifelong Model Editing (2026.findings-acl)
Copied to clipboard
Zhenghai Chen, Senbin Xu, Jiaxi Tan, Xinhua Wu, Yan Zhang, Xiawu Zheng, Shengchuan Zhang, Ke Li, Sicheng Zhao, Liujuan Cao, Rongrong Ji
| Challenge: | Existing knowledge editing methods rely on strict orthogonal projection to preserve previously edited knowledge, but this constraint limits gradient expressiveness, resulting in degradation of model generalization and overall performance as the number of edits increases. |
| Approach: | They propose a method that leverages Singular Value Decomposition to identify critical gradient subspaces and introduces a dual mechanism comprising "accumulated importance" and "projection importance" |
| Outcome: | Extensive experiments on five mainstream LLMs show that the proposed method achieves an average comprehensive performance improvement of 10.36% and effectively maintains the model’s general capabilities on downstream tasks. |