OBLIVIATE: Robust and Practical Machine Unlearning for Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) trained over corpora risk memorizing sensitive, copyrighted, or toxic content. |
| Approach: | They propose a framework that removes targeted data while preserving model utility. |
| Outcome: | The proposed framework resists membership inference attacks, minimizes impact on retained data, and maintains robustness across diverse scenarios. |
Similar Papers
Unlearn What You Want to Forget: Efficient Unlearning for LLMs (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) can be used to memorize a vast amount of data, but can suffer from privacy issues and data protection violations. |
| Approach: | They propose an efficient unlearning framework that could update LLMs without retraining them . they introduce lightweight unlearning layers learned with a selective teacher-student objective into transformers . |
| Outcome: | The proposed framework could update LLMs without having to retrain the whole model after data removals. |
Human-Inspired Obfuscation for Model Unlearning: Local and Global Strategies with Hyperbolic Representations (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for unlearning large language models struggle to balance effective forgetting with maintaining model utility. |
| Approach: | They propose a human-inspired unlearning framework that simulates forgetting on fuzzy data and represents them in hyperbolic and Euclidean spaces. |
| Outcome: | The proposed framework is able to forget sensitive content while maintaining the model’s language understanding, fluency, and benchmark performance. |
ULMR: Unlearning Large Language Models via Negative Response and Model Parameter Average (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Large language models (LLMs) have attracted significant interest from the research community due to their broad applicability in many language-oriented tasks. |
| Approach: | They propose a framework which uses pre-training datasets to rewrite instructions and generate negative responses to preserve the performance of the original LLM. |
| Outcome: | The proposed framework can erase the pre-training data while maintaining the performance of the original model. |
Decoding-Unlearning: Fact Forgetting via Entropy-Guided Inference (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for large-scale modeling memorize sensitive information . however, they are limited in real-world scenarios and require updating parameters . |
| Approach: | They propose a training-free, plug-and-play inference-time unlearning strategy that uses a probe to detect queries involving forgettable concepts and applies entropy-guided decoding to suppress target knowledge. |
| Outcome: | Experiments on MUSE, RWKU, and WMDP datasets show that SEGUE outperforms existing methods. |
Unveiling Entity-Level Unlearning for Large Language Models: A Comprehensive Analysis (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies have focused on instance-level unlearning, specifically removing predefined instances containing sensitive content. |
| Approach: | They propose a task to erase entity-related knowledge from the target model completely by analyzing the forget set and its size. |
| Outcome: | The proposed task systematically evaluates popular unlearning algorithms and reveals that the knowledge coverage of the forget set and its size play pivotal roles. |
Towards Robust Evaluation of Unlearning in LLMs via Data Transformations (2024.findings-emnlp)
Copied to clipboard
Abhinav Joshi, Shaswati Saha, Divyaksh Shukla, Sriram Vema, Harsh Jhamtani, Manas Gaur, Ashutosh Modi
| Challenge: | Large Language Models (LLMs) have shown to be a great success in a wide range of applications ranging from regular NLP-based use cases to AI agents. |
| Approach: | They examine the robustness of existing MUL techniques for their ability to enable leakage-proof forgetting in LLMs. |
| Outcome: | The proposed methods can be used to enable leakage-proof forgetting in LLMs. |
Machine Unlearning of Pre-trained Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Using curated datasets, we establish a robust benchmark for unlearning performance, demonstrating that these methods are over 105 times more computationally efficient than retraining. |
| Approach: | They propose a framework for machine unlearning in pre-trained LLMs and integrate gradient ascent with gradient descent on in-distribution data to achieve robustness. |
| Outcome: | The proposed framework is over 105 times more efficient than retraining on in-distribution data and provides detailed guidelines for efficient hyperparameter tuning in the unlearning process. |
Hey, That’s My Data! Token-Only Dataset Inference in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing dataset inference methods require logit access, but many modern LLMs restrict such access. |
| Approach: | They propose a token-only dataset inference framework that allows models to overwrite prior knowledge when trained on new data. |
| Outcome: | The proposed framework overwrites prior knowledge when trained on new data. |
Direct Token Optimization: A Self-Contained Approach to Large Language Model Unlearning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for large language models (LLMs) rely on external resources such as auxiliary models, retain datasets, or even commercial AI services. |
| Approach: | They propose a self-contained unlearning approach that optimizes the token-level objectives to unlearn specific sequences without external resources. |
| Outcome: | The proposed approach improves the forget quality up to 16.8 over the latest benchmarks while maintaining comparable model utility. |
LUNE: Efficient LLM Unlearning via LoRA Fine-Tuning with Negative Examples (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models encode vast factual knowledge, yet their inability to selectively forget specific information hinders privacy protection, bias mitigation, and post-deployment correction. |
| Approach: | They propose a LoRA-based negative-only unlearning framework that updates only low-rank adapters while freezing the backbone. |
| Outcome: | The proposed framework reduces computational cost by about an order of magnitude compared to full fine-tuning and memory-editing methods. |