Xiou Ge, Ali Mousavi, Edouard Grave, Armand Joulin, Kun Qian, Benjamin Han, Mostafa Arefiyan, Yunyao Li
| Challenge: | Existing locate-and-edit knowledge editing methods suffer from two limitations: they are infeasible for large scale KE in practice and require long run-time. |
| Approach: | They propose to use parametric fine-tuning techniques to update obsolete knowledge and induce new knowledge into LLMs. |
| Outcome: | The proposed methods improve the performance of KE and knowledge update in a temporal dataset with knowledge update and knowledge injection examples. |
Similar Papers
Knowledge Graph-Driven Memory Editing with Directional Interventions (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are hampered by inaccuracies and outdated information. |
| Approach: | They propose a framework that constructs knowledge graphs using available information to guide the direction of knowledge editing. |
| Outcome: | The proposed framework allows consistent, aligned, and stable information during large-scale editing scenarios. |
Robust Knowledge Editing via Explicit Reasoning Chains for Distractor-Resilient Multi-Hop QA (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models encode vast amounts of knowledge but remain static once trained, making timely integration of emerging facts prohibitively expensive via full retraining. |
| Approach: | They introduce a reasoning-chain-based editing framework that steers a pretrained LLM through four structured stages to filter distractors in a single pass. |
| Outcome: | The proposed framework steers a pretrained LLM through four structured stages to filter distractors in a single pass. |
Model Merging for Knowledge Editing (2025.acl-industry)
Copied to clipboard
Zichuan Fu, Xian Wu, Guojing Li, Yingying Zhang, Yefeng Zheng, Tianshi Ming, Yejing Wang, Wanyu Wang, Xiangyu Zhao
| Challenge: | Existing knowledge editing approaches struggle with sequential editing scenarios and harm the general capabilities of the model. |
| Approach: | They propose a framework that combines robust supervised fine-tuning and model merging for knowledge editing to combine supervised and supervised learning. |
| Outcome: | The proposed approach outperforms existing methods in sequential editing while preserving the original performance of the model. |
Cross-Lingual Multi-Hop Knowledge Editing (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work on knowledge editing in monolingual settings focused on a single language, but there are significant gaps in performance between the two settings. |
| Approach: | They propose a cross-lingual multi-hop knowledge editing paradigm for measuring and analyzing the performance of various SoTA knowledge editing techniques in a multilingual setup. |
| Outcome: | The proposed system improves on previous methods in a cross-lingual setting and in English. |
Accelerating LLM Fine-Tuning via Embedding Knowledge Transfer (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on parameter-efficient fine-tuning (PEFT) have produced many state-of-the-art results by adapting LLMs to new tasks, but it requires substantial training data and time to enhance model performance. |
| Approach: | They propose a parameter-efficient fine-tuning framework which efficiently transfers knowledge from a small expert model to a target large model via embedding layers. |
| Outcome: | The proposed framework accelerates domain-specific fine-tuning, improves model performance and remains robust across diverse model families and PEFT methods. |
Semantic are Beacons: A Semantic Perspective for Unveiling Parameter-Efficient Fine-Tuning in Knowledge Learning (2024.findings-acl)
Copied to clipboard
| Challenge: | Parameter-Efficient Fine-Tuning (PEFT) methods allow efficient adaptation of Large Language Models (LLMs) to various downstream tasks, but their effectiveness diminishes when downstream tasks require accurate learning of specific knowledge. |
| Approach: | They propose a method that fine-tunes a limited number of model parameters while keeping the majority of original parameters fixed. |
| Outcome: | The proposed method is able to perform on open-source large language models and validates the semantic challenge in PEFT. |
Lifelong Knowledge Editing requires Better Regularization (2025.findings-emnlp)
Copied to clipboard
Akshat Gupta, Phudish Prateepamornkul, Maochuan Lu, Ahmed Alaa, Thomas Hartvigsen, Gopala Anumanchipalli
| Challenge: | Knowledge editing is a promising way to improve factuality in large language models, but recent studies have shown significant model degradation during sequential editing. |
| Approach: | They formalize locate-then-edit methods as a two-step fine-tuning process . they show that model degradation occurs due to over-optimization of internal activations . |
| Outcome: | The proposed methods reduce time and improve factuality by 42-61%. |
Efficient Layer-wise LLM Fine-tuning for Revision Intention Prediction (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models have shown extraordinary success across text generation tasks . however, their potential for simple yet essential text classification remains underexplored . |
| Approach: | a plug-and-play layer-wise parameter-efficient fine-tuning framework is proposed . it fine- tunes a subset of important LLM layers while freezing redundant ones . |
| Outcome: | a plug-and-play framework fine-tunes a subset of important LLM layers while freezing redundant layers. |
KnowTuning: Knowledge-aware Fine-tuning for Large Language Models (2024.emnlp-main)
Copied to clipboard
Yougang Lyu, Lingyong Yan, Shuaiqiang Wang, Haibo Shi, Dawei Yin, Pengjie Ren, Zhumin Chen, Maarten Rijke, Zhaochun Ren
| Challenge: | Large language models (LLMs) are a default solution for many natural language processing tasks. |
| Approach: | They propose a knowledge-aware fine-tuning method to improve LLMs' knowledge awareness . they propose augmentation and comparison stages to improve accuracy and reliability . |
| Outcome: | The proposed method generates more facts with less factual error rate under fine-grained facts evaluation. |
Editing Across Languages: A Survey of Multilingual Knowledge Editing (2025.emnlp-main)
Copied to clipboard
| Challenge: | Knowledge Editing is a growing subdomain of model editing focused on ensuring factual edits generalize across languages. |
| Approach: | They present a taxonomy of multilingual knowledge editing methods and benchmarks . authors summarize key findings on method effectiveness and transfer patterns . |
| Outcome: | The proposed methods are compared against available benchmarks and benchmark datasets. |