Hadi Reisizadeh, Jinghan Jia, Zhiqi Bu, Bhanukiran Vinzamuri, Anil Ramakrishna, Kai-Wei Chang, Volkan Cevher, Sijia Liu, Mingyi Hong
| Challenge: | Existing algorithms to unlearn knowledge and capabilities from large datasets are unclear how to best formulate the unlearning problem. |
| Approach: | They propose to model the hierarchical structure of the unlearning problem, where the forget problem takes priority over the retain problem, and propose an algorithm that aims to unlearn knowledge and capabilities. |
| Outcome: | The proposed algorithm outperforms all state-of-the-art algorithms across unlearning tasks, models, and metrics. |
Similar Papers
Unlearn What You Want to Forget: Efficient Unlearning for LLMs (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) can be used to memorize a vast amount of data, but can suffer from privacy issues and data protection violations. |
| Approach: | They propose an efficient unlearning framework that could update LLMs without retraining them . they introduce lightweight unlearning layers learned with a selective teacher-student objective into transformers . |
| Outcome: | The proposed framework could update LLMs without having to retrain the whole model after data removals. |
Unveiling Entity-Level Unlearning for Large Language Models: A Comprehensive Analysis (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies have focused on instance-level unlearning, specifically removing predefined instances containing sensitive content. |
| Approach: | They propose a task to erase entity-related knowledge from the target model completely by analyzing the forget set and its size. |
| Outcome: | The proposed task systematically evaluates popular unlearning algorithms and reveals that the knowledge coverage of the forget set and its size play pivotal roles. |
UNLEARN Efficient Removal of Knowledge in Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large Language Models excel in many tasks but are outperformed by specialized tools for certain tasks. |
| Approach: | They propose a method that uses subspace techniques to selectively remove knowledge . they propose 'unlearn' method that can forget or unlear the knowledge without retraining . |
| Outcome: | The proposed method outperforms existing methods for forgetting target knowledge while preserving related knowledge. |
Reveal and Release: Iterative LLM Unlearning with Self-generated Data (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to unlearning large language models assume full access to the forget dataset, overlooking two key challenges: (1) Forget data is often privacy-sensitive, rare, or legally regulated, making it expensive or impractical to obtain (2) The distribution of available forget data may not align with how that information is represented within the model. |
| Approach: | They propose a “Reveal-and-Release” method to unlearn with self-generated data, prompting the model to reveal what it knows using optimized instructions. |
| Outcome: | The proposed method removes the influence of undesirable data from the model. |
UIPE: Enhancing LLM Unlearning by Removing Knowledge Related to Forgetting Targets (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to unlearning large language models focus on forgetting target data while overlooking the impact of logically related knowledge on the effectiveness of unlearning. |
| Approach: | They propose a method that removes knowledge highly correlated with the forgetting targets and a technique that remove logically related knowledge from the model. |
| Outcome: | The proposed method significantly improves the performance of the proposed method on the TOFU and WMDP benchmarks. |
Unlearners Can Lie: Evaluating and Improving Honesty in LLM Unlearning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for unlearning in large language models often hallucinate, generate abnormal token sequences, or behave inconsistently, raising safety and trust concerns. |
| Approach: | They propose a formal definition of unlearning honesty that preserves both utility and honesty on retained knowledge and ensures effective forgetting while encouraging the model to acknowledge its limitations. |
| Outcome: | The proposed method achieves highest rejection rate and refusal stability on Q A tasks from the forget set, nearly double the second-best method. |
Representation-Guided Parameter-Efficient LLM Unlearning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to unlearning large language models often memorize sensitive or harmful information, but they struggle with the forget-retain trade-off due to the polysemantic nature of LLMs parameters. |
| Approach: | They propose a representation-guided low-rank unlearning approach that leverages the geometric properties of representation spaces to achieve robust and precise unlearning. |
| Outcome: | The proposed approach outperforms state-of-the-art models on TOFU and WMDP benchmarks while maintaining higher model utility. |
Human-Inspired Obfuscation for Model Unlearning: Local and Global Strategies with Hyperbolic Representations (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for unlearning large language models struggle to balance effective forgetting with maintaining model utility. |
| Approach: | They propose a human-inspired unlearning framework that simulates forgetting on fuzzy data and represents them in hyperbolic and Euclidean spaces. |
| Outcome: | The proposed framework is able to forget sensitive content while maintaining the model’s language understanding, fluency, and benchmark performance. |
Selective Span-Level Unlearning for Large Language Models (2026.acl-short)
Copied to clipboard
| Challenge: | Existing selective methods that focus on identifying token-level or span-level unlearning targets are misaligning unlearning objectives with the model’s internal behavior. |
| Approach: | They propose a selective method that uses model-intrinsic information to identify token-level or span-level unlearning targets within a text rather than entire sequences. |
| Outcome: | The proposed method achieves comparable unlearning performance while significantly better preserving retained knowledge. |
Towards Robust Evaluation of Unlearning in LLMs via Data Transformations (2024.findings-emnlp)
Copied to clipboard
Abhinav Joshi, Shaswati Saha, Divyaksh Shukla, Sriram Vema, Harsh Jhamtani, Manas Gaur, Ashutosh Modi
| Challenge: | Large Language Models (LLMs) have shown to be a great success in a wide range of applications ranging from regular NLP-based use cases to AI agents. |
| Approach: | They examine the robustness of existing MUL techniques for their ability to enable leakage-proof forgetting in LLMs. |
| Outcome: | The proposed methods can be used to enable leakage-proof forgetting in LLMs. |