Finding Memo: Extractive Memorization in Constrained Sequence Generation Tasks (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Memorization presents a challenge for constrained Natural Language Generation tasks . previous studies focused on counterfactual memorization, linking it to hallucinations . |
| Approach: | They propose an algorithm for extractive memorization in constrained sequence generation tasks . they propose to elicit non-memorized translations of memorized samples from the same model . |
| Outcome: | The proposed algorithm could be leveraged to mitigate memorization in the model through finetuning. |
Similar Papers
Unlocking Memorization in Large Language Models with Dynamic Soft Prompting (2024.emnlp-main)
Copied to clipboard
Zhepeng Wang, Runxue Bao, Yawen Wu, Jackson Taylor, Cao Xiao, Feng Zheng, Weiwen Jiang, Shangqian Gao, Yanfu Zhang
| Challenge: | Pretrained large language models excel in a variety of natural language processing tasks . however, they pose significant security risks due to their tendency to memorize training data . |
| Approach: | They propose a method to estimate LLM memorization using dynamic, prefix-dependent soft prompts. |
| Outcome: | The proposed method can achieve maximum relative improvement of 135.3% and 39.8% over baseline compared to state-of-the-art methods. |
Bridging the Gap between Training and Inference for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Neural Machine Translation generates target words sequentially while at inference it has to generate the entire sequence from scratch. |
| Approach: | They propose to use ground truth and inference to generate target words sequentially while at inference it has to generate the entire sequence from scratch. |
| Outcome: | Experiments on Chinese->English and WMT’14 English->German translation tasks show that the proposed model can achieve significant improvements on multiple datasets. |
Retracing the Past: LLMs Emit Training Data When They Get Lost (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for extracting training data from large language models exhibit limited success . existing methods offer limited insight into the fundamental drivers of memorization leakage . |
| Approach: | They propose a framework for extracting memorized data by maximizing model uncertainty . they propose mismatched fine-tuning to weaken alignment and induce confusion . |
| Outcome: | The proposed attacks outperform baselines on unaligned and aligned LLMs . the proposed attacks exploit the model uncertainty of the input snippets induced by the model entropy spike . |
Measuring memorization in language models via probabilistic extraction (2025.naacl-long)
Copied to clipboard
Jamie Hayes, Marika Swanberg, Harsh Chaudhari, Itay Yona, Ilia Shumailov, Milad Nasr, Christopher A. Choquette-Choo, Katherine Lee, A. Feder Cooper
| Challenge: | Large language models (LLMs) are susceptible to memorizing training data, raising concerns about the potential extraction of sensitive information at generation time. |
| Approach: | They propose a method that splits training example into prefix and suffix, prompts LLM with suffix and deems it extractable if it generates the suffix using greedy sampling. |
| Outcome: | The proposed method is unreliable because it does not account for non-determinism in more realistic sampling schemes. |
Get Confused Cautiously: Textual Sequence Memorization Erasure with Selective Entropy Maximization (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for erasure of memorized text fail to unlearn large numbers of memorizable samples without jeopardizing model utility. |
| Approach: | They propose a method that allows LLMs to memorize and recite some training sequences verbatim . they propose an entropy-based loss method that is shown to be more stable . |
| Outcome: | The proposed method improves model utility and accuracy while preserving model ability in language generation and understanding. |
An Empirical Analysis of Memorization in Fine-tuned Autoregressive Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Large language models are shown to present privacy risks through memorization of training data, but little attention has been given to the fine-tuning phase. |
| Approach: | They empirically study memorization of fine-tuning methods using membership inference and extraction attacks and show that fine-timing the head of the model has the highest susceptibility to attacks. |
| Outcome: | The proposed methods have the highest memorization risk, whereas the smaller adapters are less vulnerable to known extraction attacks. |
Memorisation Cartography: Mapping out the Memorisation-Generalisation Continuum in Neural Machine Translation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Using the counterfactual memorisation metric, we find that when training neural networks, models will memorise some inputs but not others. |
| Approach: | They use the counterfactual memorisation metric to build a resource that places 5M NMT datapoints on a memorisations-generalisation map and describe how the datapoint’s surface-level characteristics and a models’ per-datum training signals are predictive of memorising in NMT. |
| Outcome: | The proposed model places 5M NMT datapoints on a memorisation-generalisation map and shows how their surface-level characteristics and models’ per-datum training signals are predictive of memorising in NMT. |
Controlling the Extraction of Memorized Data from Large Language Models via Prompt-Tuning (2023.acl-short)
Copied to clipboard
Mustafa Ozdayi, Charith Peris, Jack FitzGerald, Christophe Dupuy, Jimit Majmudar, Haidar Khan, Rahil Parikh, Rahul Gupta
| Challenge: | Large Language Models memorize significant portions of training data, which poses privacy risk. |
| Approach: | They propose a prompt-tuning approach to control the extraction rates of memorized content in large language models. |
| Outcome: | The proposed techniques yield 9.3% increase in extraction rate compared to baseline model . the proposed defense achieves 97.7% reduction with a perplexity increase of 16.9% . |
Fine-grained Contrastive Learning for Definition Generation (2022.aacl-main)
Copied to clipboard
| Challenge: | Recent pre-trained transformer-based definition generation models lack effective representation learning to contain full semantic components of the given word, leading to under-specific definitions. |
| Approach: | They propose a novel contrastive learning method that encourages the model to capture more detailed semantic representations from the definition sequence encoding. |
| Outcome: | The proposed method could generate more specific definitions compared with state-of-the-art models. |
Categorizing Semantic Representations for Neural Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Modern neural machine translation models suffer limitation in compositional generalization, resulting in weakened translation performance on unseen compounds. |
| Approach: | They propose to introduce categorization to the contextualized representations to improve generalization by reducing sparsity and overfitting. |
| Outcome: | The proposed method reduces compositional generalization error rates by 24% on a dedicated MT dataset. |