Auditing Language Model Unlearning via Information Decomposition (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing approaches to unlearning in language models do not account for information about forgotten data . despite the apparent success of unlearning, information about the forgotten data remains linearly decodable from internal representations. |
| Approach: | They propose an interpretable framework for auditing unlearning using Partial Information Decomposition . they propose a representation-based risk score that can guide abstention on sensitive inputs . |
| Outcome: | The proposed framework can guide abstention on sensitive inputs at inference time. |
Similar Papers
Cross-Lingual Unlearning of Selective Knowledge in Multilingual Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained language models memorize large amounts of information, raising significant safety concerns. |
| Approach: | They propose an approach to machine unlearning for multilingual language models that selectively erases information across different languages while maintaining overall performance. |
| Outcome: | The proposed approach is compared with existing unlearning baselines and set a new standard for secure and adaptable multilingual language models. |
Towards Robust Evaluation of Unlearning in LLMs via Data Transformations (2024.findings-emnlp)
Copied to clipboard
Abhinav Joshi, Shaswati Saha, Divyaksh Shukla, Sriram Vema, Harsh Jhamtani, Manas Gaur, Ashutosh Modi
| Challenge: | Large Language Models (LLMs) have shown to be a great success in a wide range of applications ranging from regular NLP-based use cases to AI agents. |
| Approach: | They examine the robustness of existing MUL techniques for their ability to enable leakage-proof forgetting in LLMs. |
| Outcome: | The proposed methods can be used to enable leakage-proof forgetting in LLMs. |
Unveiling Entity-Level Unlearning for Large Language Models: A Comprehensive Analysis (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies have focused on instance-level unlearning, specifically removing predefined instances containing sensitive content. |
| Approach: | They propose a task to erase entity-related knowledge from the target model completely by analyzing the forget set and its size. |
| Outcome: | The proposed task systematically evaluates popular unlearning algorithms and reveals that the knowledge coverage of the forget set and its size play pivotal roles. |
Knowledge Unlearning for Mitigating Privacy Risks in Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Recent work shows that an adversary can extract training data from Pretrained Language Models including Personally Identifiable Information (PII) such as names, phone numbers, and email addresses. |
| Approach: | They propose to use knowledge unlearning to reduce privacy risks for LMs by performing gradient ascent on target token sequences instead of trying to unlearn all the data at once. |
| Outcome: | The proposed method can give a stronger empirical privacy guarantee in scenarios where the data vulnerable to extraction attacks are known a priori while being much more efficient and robust. |
ReLearn: Unlearning via Learning for Large Language Models (2025.acl-long)
Copied to clipboard
Haoming Xu, Ningyuan Zhao, Liming Yang, Sendong Zhao, Shumin Deng, Mengru Wang, Bryan Hooi, Nay Oo, Huajun Chen, Ningyu Zhang
| Challenge: | Existing methods for unlearning large language models often rely on reverse optimization to reduce target token probabilities. |
| Approach: | They propose a data augmentation and fine-tuning pipeline for effective unlearning . they propose augmentation, evaluation frameworks to measure contextual forgetting . |
| Outcome: | The proposed framework achieves targeted forgetting while preserving high-quality outputs. |
Erasing Without Remembering: Implicit Knowledge Forgetting in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | a new method for unlearning large language models is proposed to improve the performance of large language model models. |
| Approach: | They propose a probability perturbation-based unlearning paradigm that allows models to forget implicit knowledge in large language models with a focus on generalisation. |
| Outcome: | The proposed model improves unlearning vanilla target data while forgetting implicit knowledge. |
Identifying Unlearned Data in LLMs via Membership Inference Attacks (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing work evaluates approximate unlearning under a retrieval paradigm, where adversaries attempt to extract residual knowledge given partial information of the unlearning target. |
| Approach: | They propose a framework to evaluate unlearning membership attacks using member inference techniques to exploit the forget set. |
| Outcome: | The proposed framework assesses whether unlearning leaves behind detectable artifacts that can be exploited to infer membership in the forget set. |
Machine Unlearning of Pre-trained Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Using curated datasets, we establish a robust benchmark for unlearning performance, demonstrating that these methods are over 105 times more computationally efficient than retraining. |
| Approach: | They propose a framework for machine unlearning in pre-trained LLMs and integrate gradient ascent with gradient descent on in-distribution data to achieve robustness. |
| Outcome: | The proposed framework is over 105 times more efficient than retraining on in-distribution data and provides detailed guidelines for efficient hyperparameter tuning in the unlearning process. |
Soft Token Attacks Cannot Reliably Audit Unlearning in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work shows that soft token attacks can extract unlearned information from large language models. |
| Approach: | They show that soft token attacks can extract unlearned information from LLMs . |
| Outcome: | The proposed attacks can extract unlearned information from large language models . |
Decoding-Unlearning: Fact Forgetting via Entropy-Guided Inference (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for large-scale modeling memorize sensitive information . however, they are limited in real-world scenarios and require updating parameters . |
| Approach: | They propose a training-free, plug-and-play inference-time unlearning strategy that uses a probe to detect queries involving forgettable concepts and applies entropy-guided decoding to suppress target knowledge. |
| Outcome: | Experiments on MUSE, RWKU, and WMDP datasets show that SEGUE outperforms existing methods. |