| Challenge: | Large Language Models (LLMs) have demonstrated incredible capabilities in understanding, generating, and manipulating languages. |
| Approach: | They propose a general SParse Efficient Editing MoDel which can fulfill diverse editing requirements through a single model while maintaining low computational costs. |
| Outcome: | The proposed model can fulfill diverse editing requirements through a single model while maintaining low computational costs. |
Similar Papers
Editing Large Language Models: Problems, Methods, and Opportunities (2023.emnlp-main)
Copied to clipboard
Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, Ningyu Zhang
| Challenge: | Recent advances in model editing for LLMs have created challenges and opportunities for the community. |
| Approach: | They propose to alter the behavior of LLMs efficiently within a specific domain without negatively impacting performance across other inputs. |
| Outcome: | The proposed method alters behavior of LLMs efficiently within a specific domain without negatively impacting performance across other inputs. |
Knowledge Editing for Large Language Models (2024.lrec-tutorials)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are not immune to issues of factual accuracy or logically consistent. |
| Approach: | This tutorial will present cutting-edge methods and practical tools for editing Large Language Models (LLMs). |
| Outcome: | The aim of this course is to familiarize researchers with the latest advancements and emerging strategies in the realm of knowledge editing for LLMs. |
A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Sparse Autoencoders (SAEs) can disentangle complex features into more interpretable components. |
| Approach: | They propose to use Sparse Autoencoders to disentangle LLM features into more interpretable components. |
| Outcome: | The proposed method disentangles complex features into more interpretable components. |
Exploiting Edited Large Language Models as General Scientific Optimizers (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for solving optimization problems in scientific scenarios use observational feedback as additional textual descriptions, but these methods struggle to utilize it effectively. |
| Approach: | They propose a generalized approach to boost mathematical optimization in scientific scenarios by using observational feedback from LLMs as additional textual descriptions. |
| Outcome: | The proposed method outperforms existing state-of-the-art methods on six different tasks using six different LLM backbones. |
Bridging the Editing Gap in LLMs: FineEdit for Precise and Targeted Text Modifications (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that large language models can perform precise text editing tasks. |
| Approach: | InstrEditBench is a benchmark dataset that compares 30,000 structured editing tasks . experimental evaluations show FineEdit outperforms state-of-the-art models . |
| Outcome: | The proposed model outperforms state-of-the-art models on single-turn edits and mistral-7B-OpenOrca on direct edits. |
Robust and Scalable Model Editing for Large Language Models (2024.lrec-main)
Copied to clipboard
Yingfa Chen, Zhengyan Zhang, Xu Han, Chaojun Xiao, Zhiyuan Liu, Chen Chen, Kuai Li, Tao Yang, Maosong Sun
| Challenge: | Existing methods that ignore contextual knowledge fail to reliably fall back to parametric knowledge when presented with irrelevant context. |
| Approach: | They propose to use contextual knowledge to update and correct LLMs' knowledge by in-context editing instead of retraining. |
| Outcome: | The proposed method outperforms current state-of-the-art methods by a large margin on a dataset that contains irrelevant questions. |
Improving Cross-Domain Low-Resource Text Generation through LLM Post-Editing: A Programmer-Interpreter Approach (2024.findings-eacl)
Copied to clipboard
| Challenge: | Large pre-trained language models such as GPT-3.5 and GPT-4 have gained significant attention in natural language research due to limited computational resources or inaccessible parameters. |
| Approach: | They propose a neural programmer-interpreter approach that preserves the domain generalization ability of LLMs while editing their output. |
| Outcome: | The proposed framework significantly improves GPT-3.5’s performance in logical form-to-text conversion and low-resource machine translation, surpassing other state-of-the-art (SOTA) LLM post-editing methods in cross-domain settings. |
AdaEdit: Advancing Continuous Knowledge Editing For Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge editing methods that can efficiently update knowledge in LLMs are limited due to budget constraints. |
| Approach: | They propose a method that can enhance the performance of edited LLMs in large-size continuous editing regimes. |
| Outcome: | Extensive empirical evaluations on multiple LLMs show that the proposed method outperforms existing methods without compromising the general abilities of these models. |
InstructCoder: Instruction Tuning Large Language Models for Code Editing (2024.acl-srw)
Copied to clipboard
| Challenge: | InstructCoder is the first instruction-tuning dataset designed to adapt LLMs for general-purpose code editing. |
| Approach: | They propose to use Large Language Models to edit code based on user instructions . they use a dataset to adapt LLMs to general-purpose code editing . |
| Outcome: | The proposed model can significantly improve code editing performance compared to proprietary models . the proposed model is based on a human-written execution-based benchmark . |
Enhancing Semantic Consistency of Large Language Models through Model Editing: An Interpretability-Oriented Approach (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models generate inconsistent and sometimes contradictory outputs when presented with a prompt that has equivalent semantics but is expressed differently from the original prompt. |
| Approach: | They propose to refine a Large Language Model (LLM) with prompt-output pairs with equivalent semantics to achieve semantic consistency. |
| Outcome: | The proposed method improves the semantic consistency and task performance of LLMs. |