R.R.: Unveiling LLM Training Privacy through Recollection and Ranking (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing privacy attacks focus on membership inference or data extraction, but reconstructing specific personally identifiable information (PII) in training data remains challenging. |
| Approach: | They propose a two-step privacy stealing attack that enables attackers to reconstruct PII entities from scrubbed training data where the PI I entities have been masked. |
| Outcome: | The proposed attack can reconstruct PII entities from scrubbed training data where the PI I entities have been masked. |
Similar Papers
Do LLMs Really Memorize Personally Identifiable Information? Revisiting PII Leakage with a Cue-Controlled Memorization Framework (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been reported to “leak” Personally Identifiable Information (PII) successful PII reconstruction often interpreted as evidence of memorization. |
| Approach: | They propose a principled revision of memorization evaluation for Large Language Models . they propose PII leakage should be evaluated under low lexical cue conditions . |
| Outcome: | The proposed method is based on a multilingual re-evaluation of PII leakage across 32 languages and multiple memorization paradigms. |
Exploiting the Shadows: Unveiling Privacy Leaks through Lower-Ranked Tokens in Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models face vulnerabilities related to the extraction of sensitive information. |
| Approach: | They propose a method to exploit the model's lower-ranked output tokens to extract private information from retrieved documents or training knowledge. |
| Outcome: | The proposed method is effective in both the agentic application privacy extraction setting and the direct training data extraction. |
Learning to Refuse: Towards Mitigating Privacy Risks in LLMs (2025.coling-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit remarkable capabilities in understanding and generating natural languages, but can inadvertently memorize private information, posing significant privacy risks. |
| Approach: | They propose to use a dataset to evaluate machine unlearning methods for protecting personal data in a realistic scenario. |
| Outcome: | The proposed model outperforms baseline methods by 5.65 points and protects target individuals’ personal data while maintaining general capabilities. |
Tokens for Learning, Tokens for Unlearning: Mitigating Membership Inference Attacks in Large Language Models via Dual-Purpose Training (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing defenses for large language models do not account for the sequential nature of text data. |
| Approach: | They propose a lightweight yet effective empirical privacy defense that leverages token-specific characteristics to protect training data of large language models. |
| Outcome: | The proposed approach provides strong protection against membership inference attacks and improves language modeling performance by 10% across different LLM architectures and datasets compared to baselines. |
Identifying Unlearned Data in LLMs via Membership Inference Attacks (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing work evaluates approximate unlearning under a retrieval paradigm, where adversaries attempt to extract residual knowledge given partial information of the unlearning target. |
| Approach: | They propose a framework to evaluate unlearning membership attacks using member inference techniques to exploit the forget set. |
| Outcome: | The proposed framework assesses whether unlearning leaves behind detectable artifacts that can be exploited to infer membership in the forget set. |
Retracing the Past: LLMs Emit Training Data When They Get Lost (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for extracting training data from large language models exhibit limited success . existing methods offer limited insight into the fundamental drivers of memorization leakage . |
| Approach: | They propose a framework for extracting memorized data by maximizing model uncertainty . they propose mismatched fine-tuning to weaken alignment and induce confusion . |
| Outcome: | The proposed attacks outperform baselines on unaligned and aligned LLMs . the proposed attacks exploit the model uncertainty of the input snippets induced by the model entropy spike . |
Unintended Memorization of Sensitive Information in Fine-Tuned Language Models (2026.eacl-long)
Copied to clipboard
Marton Szep, Jorge Marin Ruiz, Georgios Kaissis, Paulina Seidl, Rüdiger von Eisenhart-Rothe, Florian Hinterwimmer, Daniel Rueckert
| Challenge: | Large Language Models (LLMs) on sensitive datasets carry a substantial risk of unintended memorization and leakage of Personally Identifiable Information (PII) prior studies have analyzed memorizing dynamics in LLMs during pre-training and fine-tuning. |
| Approach: | They investigate the vulnerability of PII that appears only in model inputs, not in training targets. |
| Outcome: | The proposed methods show that post-training methods provide more consistent privacy-utility trade-offs . |
Private Memorization Editing: Turning Memorization into a Defense to Strengthen Data Privacy in Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) memorize and therefore, among huge amounts of uncontrolled data, may memorize Personally Identifiable Information (PII). |
| Approach: | They propose a method that uses a model knowledge to memorize PII from training data to mitigate the memorization of PI I. |
| Outcome: | The proposed method reduces the number of leaked PIIs in a number of configurations while making it more robust against privacy Training Data Extraction attacks. |
REVS: Unlearning Sensitive Information in Language Models via Rank Editing in the Vocabulary Space (2025.findings-acl)
Copied to clipboard
| Challenge: | Current approaches to address this issue involve costly dataset scrubbing or model filtering through unlearning and model editing. |
| Approach: | They propose a method for unlearning sensitive information from language models . they curate email and URL datasets and a social security number dataset . |
| Outcome: | The proposed method shows superior performance and robustness to extraction attacks on real-world datasets. |
Stealing Training Data from Large Language Models in Decentralized Training through Activation Inversion Attack (2025.acl-long)
Copied to clipboard
| Challenge: | Decentralized training is a resource-efficient framework to democratize training of large language models. |
| Approach: | They propose an activation inversion attack to exploit privacy leakage from training data . they construct a shadow dataset comprising text labels and corresponding activations . |
| Outcome: | The proposed attack surface is based on a shadow dataset and public datasets . the proposed attack model reconstructs training data from activations in victim decentralized training. |