The Silence of the Facts: Popularity as a Barrier to Machine Unlearning (2026.acl-srw)
Copied to clipboard
| Challenge: | Existing unlearning methods assume that all facts are equally challenging to forget . large models struggle more to forget popular entities, damaging related knowledge in the process . |
| Approach: | They build a benchmark to investigate whether fact popularity influences the efficiency of LLM unlearning. |
| Outcome: | The proposed benchmark compares state-of-the-art models on a set of models of different sizes. |
Similar Papers
UNLEARN Efficient Removal of Knowledge in Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large Language Models excel in many tasks but are outperformed by specialized tools for certain tasks. |
| Approach: | They propose a method that uses subspace techniques to selectively remove knowledge . they propose 'unlearn' method that can forget or unlear the knowledge without retraining . |
| Outcome: | The proposed method outperforms existing methods for forgetting target knowledge while preserving related knowledge. |
Unlearn What You Want to Forget: Efficient Unlearning for LLMs (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) can be used to memorize a vast amount of data, but can suffer from privacy issues and data protection violations. |
| Approach: | They propose an efficient unlearning framework that could update LLMs without retraining them . they introduce lightweight unlearning layers learned with a selective teacher-student objective into transformers . |
| Outcome: | The proposed framework could update LLMs without having to retrain the whole model after data removals. |
Unlearning vs. Obfuscation: Are We Truly Removing Knowledge? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent methods often rely on obfuscation by injecting incorrect or irrelevant information to suppress knowledge, leaving models vulnerable to probing. |
| Approach: | They propose a method that flattens the model predictive distribution over automatically generated multiple-choice questions, effectively removing knowledge about target individuals. |
| Outcome: | The proposed method achieves unlearning with over 90% refusal rate and a higher uncertainty than obfuscation on probing questions. |
Decoding-Unlearning: Fact Forgetting via Entropy-Guided Inference (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for large-scale modeling memorize sensitive information . however, they are limited in real-world scenarios and require updating parameters . |
| Approach: | They propose a training-free, plug-and-play inference-time unlearning strategy that uses a probe to detect queries involving forgettable concepts and applies entropy-guided decoding to suppress target knowledge. |
| Outcome: | Experiments on MUSE, RWKU, and WMDP datasets show that SEGUE outperforms existing methods. |
Which Retain Set Matters for LLM Unlearning? A Case Study on Entity Unlearning (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are prone to retaining unauthorized or sensitive information from their training data, which raises privacy concerns. |
| Approach: | They propose to use a group of queries that share similar syntactic structures with the data targeted for removal to investigate the effects of unlearning on various subsets of the retain set. |
| Outcome: | The proposed method reduces the retention set, the portion of training data that is not targeted for removal, and improves model performance across subsets. |
Unveiling Entity-Level Unlearning for Large Language Models: A Comprehensive Analysis (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies have focused on instance-level unlearning, specifically removing predefined instances containing sensitive content. |
| Approach: | They propose a task to erase entity-related knowledge from the target model completely by analyzing the forget set and its size. |
| Outcome: | The proposed task systematically evaluates popular unlearning algorithms and reveals that the knowledge coverage of the forget set and its size play pivotal roles. |
Machine Unlearning of Pre-trained Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Using curated datasets, we establish a robust benchmark for unlearning performance, demonstrating that these methods are over 105 times more computationally efficient than retraining. |
| Approach: | They propose a framework for machine unlearning in pre-trained LLMs and integrate gradient ascent with gradient descent on in-distribution data to achieve robustness. |
| Outcome: | The proposed framework is over 105 times more efficient than retraining on in-distribution data and provides detailed guidelines for efficient hyperparameter tuning in the unlearning process. |
Not Every Token Needs Forgetting: Selective Unlearning Balancing Forgetting and Utility in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Conventional unlearning approaches forget all tokens in a target document, including common tokens that carry general knowledge. |
| Approach: | They propose a method that identifies a critical subset of tokens within the forgetting set that is relevant to the unwanted information and unlearns only those tokens. |
| Outcome: | Experiments on two benchmarks and six baseline unlearning algorithms show that selective unlearning achieves effective unlearning on the targeted forget data. |
Learning to Refuse: Towards Mitigating Privacy Risks in LLMs (2025.coling-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit remarkable capabilities in understanding and generating natural languages, but can inadvertently memorize private information, posing significant privacy risks. |
| Approach: | They propose to use a dataset to evaluate machine unlearning methods for protecting personal data in a realistic scenario. |
| Outcome: | The proposed model outperforms baseline methods by 5.65 points and protects target individuals’ personal data while maintaining general capabilities. |
UIPE: Enhancing LLM Unlearning by Removing Knowledge Related to Forgetting Targets (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to unlearning large language models focus on forgetting target data while overlooking the impact of logically related knowledge on the effectiveness of unlearning. |
| Approach: | They propose a method that removes knowledge highly correlated with the forgetting targets and a technique that remove logically related knowledge from the model. |
| Outcome: | The proposed method significantly improves the performance of the proposed method on the TOFU and WMDP benchmarks. |