Analyzing Memorization in Large Language Models through the Lens of Model Attribution (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing research has focused on extracting memorized content from LLMs or developing memorization metrics without exploring the underlying architectural factors that contribute to memorizing. |
| Approach: | They analyze how attention modules at different layers impact its memorization and generalization performance by using attribution techniques. |
| Outcome: | The proposed model can be used to mitigate memorization while keeping other components like layer normalization and MLP transformations intact. |
Similar Papers
Exploring Memorization in Fine-tuned Language Models (2024.acl-long)
Copied to clipboard
Shenglai Zeng, Yaxin Li, Jie Ren, Yiding Liu, Han Xu, Pengfei He, Yue Xing, Shuaiqiang Wang, Jiliang Tang, Dawei Yin
| Challenge: | Existing studies have shown that pre-trained langauge models tend to memorize and regenerate segments of their pre-training corpus when prompted appropriately. |
| Approach: | They conduct the first comprehensive analysis to explore language models’ memorization during fine-tuning across tasks. |
| Outcome: | The proposed analysis shows that memorization presents a strong disparity among different fine-tuning tasks. |
Controllable Memorization in LLMs via Weight Pruning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have focused on mitigating memorization, but the deliberate control of memorisation has been underexplored. |
| Approach: | They propose a gradient-based weight pruning framework to control memorization rates in large language models by fine-grained control over pruning parameters. |
| Outcome: | The proposed framework enables models to suppress or enhance memorization based on application-specific requirements. |
Memorization ≠ Understanding: Do Large Language Models Have the Ability of Scenario Cognition? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated impressive performance across NLP tasks. |
| Approach: | They propose a framework to assess LLMs’ scenario cognition . they examine the ability to link semantic scenario elements with their arguments in context . |
| Outcome: | The proposed framework assesses large language models’ scenario cognition . it shows that current models rely on superficial memorization, failing to achieve robust semantic scenario cognition even in simple cases. |
Neuron-Level Differentiation of Memorization and Generalization in Large Language Models (2025.emnlp-main)
Copied to clipboard
Ko-Wei Huang, Yi-Fu Fu, Ching-Yu Tsai, Yu-Chieh Tu, Tzu-ling Cheng, Cheng-Yu Lin, Yi-Ting Yang, Heng-Yi Liu, Keng-Te Liao, Da-Cheng Juan, Shou-De Lin
| Challenge: | Existing models exhibit memorization and generalization behaviors in ways that are not easily interpretable or controllable. |
| Approach: | They propose to use a GPT-2 and LLaMA-3.2 model to identify distinct neuron subsets responsible for each behavior to steer the model toward memorization or generalization. |
| Outcome: | The proposed models show that inference-time interventions on these neurons can steer the model’s behavior toward memorization or generalization. |
Automatic Evaluation of Attribution by Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Generative large language models (LLMs) incorporate external references to generate and support claims. however, evaluating the attribution remains an open problem. |
| Approach: | They investigate automatic evaluation of attribution given by large language models . they define different types of attributed errors and then explore two approaches . |
| Outcome: | The proposed methods highlight promising signals and challenges. |
Demystifying Verbatim Memorization in Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that Large Language Models (LLMs) memorize long sequences verbatim, with serious copyright and privacy implications. |
| Approach: | They develop a framework to study verbatim memorization in a controlled setting by continuing pre-training from Pythia checkpoints with injected sequences. |
| Outcome: | The proposed framework creates a control model M () and a treatment model M with injected sequences. |
Unintended Memorization of Sensitive Information in Fine-Tuned Language Models (2026.eacl-long)
Copied to clipboard
Marton Szep, Jorge Marin Ruiz, Georgios Kaissis, Paulina Seidl, Rüdiger von Eisenhart-Rothe, Florian Hinterwimmer, Daniel Rueckert
| Challenge: | Large Language Models (LLMs) on sensitive datasets carry a substantial risk of unintended memorization and leakage of Personally Identifiable Information (PII) prior studies have analyzed memorizing dynamics in LLMs during pre-training and fine-tuning. |
| Approach: | They investigate the vulnerability of PII that appears only in model inputs, not in training targets. |
| Outcome: | The proposed methods show that post-training methods provide more consistent privacy-utility trade-offs . |
ReGLA: Refining Gated Linear Attention (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) are known for their computational and storage requirements due to the quadratic computation complexity of softmax attention. |
| Approach: | They propose to reduce the quadratic computation complexity of softmax attention by using feature maps, normalization and the gating mechanism to improve performance. |
| Outcome: | The proposed model outperforms existing gated linear attention models in extensive tasks including training from scratch and post-linearization with continual pre-training. |
A Multi-Perspective Analysis of Memorization in Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can generate the same sequences contained in the pre-train corpus, known as memorization. |
| Approach: | They analyze the relationship between memorization and outputs from Large Language Models (LLMs) they show a sudden drop and increase in the frequency of input tokens when generating memorized/unmemorized sequences . |
| Outcome: | The proposed model can generate the same sequences contained in the pre-train corpus, and it can predict unmemorized tokens. |
Toward Inclusive Language Models: Sparsity-Driven Calibration for Systematic and Interpretable Mitigation of Social Biases in LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new method to mitigate stereotypical bias in large language models is needed . inherent biases from training on vast Internet datasets can amplify harmful stereotypes . |
| Approach: | They propose a method to identify stereotypical bias in decoder-only transformer models . they apply a localization mechanism that correlates internal activations with a new Context Influence score . |
| Outcome: | The proposed method reduces stereotypical biases on BBQ, StereoSet, and CrowS-Pairs while improving reasoning performance on MMLU by 10%. |