| Challenge: | Large pretrained language models (LLMs) have shown surprising In-Context Learning ability. |
| Approach: | They propose a locally differentially private framework of in-context learning for LLMs that can be augmented with a private database for some specific task. |
| Outcome: | The proposed framework can predict labels without additional parameter modifications without input-label pairs . |
Similar Papers
Public Data Assisted Differentially Private In-Context Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning has shown remarkable performance across tasks without fine-tuning . however, recent studies have highlighted the risk of private data leakage through the prompt in ICL . |
| Approach: | They propose a private in-context learning algorithm that effectively balances privacy protection and model utility. |
| Outcome: | The proposed algorithm is robust against membership inference attacks and is robust to membership infertility attacks. |
Learning and Evaluating a Differentially Private Pre-trained Language Model (2021.findings-emnlp)
Copied to clipboard
Shlomo Hoory, Amir Feder, Avichai Tendler, Sofia Erell, Alon Peled-Cohen, Itay Laish, Hootan Nakhost, Uri Stemmer, Ayelet Benjamini, Avinatan Hassidim, Yossi Matias
| Challenge: | Contextual language models have improved performance but can lead to information leakage . |
| Approach: | They propose a differentially-private word-piece algorithm that allows training a tailored domain-specific vocabulary while maintaining privacy. |
| Outcome: | The proposed model can guarantee privacy while maintaining good model performance. |
Differentially Private Language Models for Secure Data Sharing (2022.emnlp-main)
Copied to clipboard
| Challenge: | a variety of deanonymization attacks allow the re-identification of individuals from tabular data. |
| Approach: | They propose to train a language model in a differentially private manner and sample data from it . they find that the model generates fluent textual datasets with privacy guarantees . |
| Outcome: | The proposed methods outperform direct classifiers with DP-SGD in the real-world. |
Training Text-to-Text Transformers with Privacy Guarantees (2022.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in NLP often stem from large transformer-based pre-trained models. |
| Approach: | They propose differentially private (DP) training as a potential mitigation for models that can memorize parts of training data. |
| Outcome: | The proposed model can memorize parts of training data and mitigate memorization concerns. |
Large Language Models Can Be Contextual Privacy Protection Learners (2024.emnlp-main)
Copied to clipboard
Yijia Xiao, Yiqiao Jin, Yushi Bai, Yue Wu, Xianjun Yang, Xiao Luo, Wenchao Yu, Xujiang Zhao, Yanchi Liu, Quanquan Gu, Haifeng Chen, Wei Wang, Wei Cheng
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable linguistic comprehension and generation capability, but when applied to specialized industries, they face challenges such as hallucination, insufficient domain knowledge, and failing to incorporate the latest domain knowledge. |
| Approach: | They propose a paradigm for fine-tuning LLMs that effectively injects domain-specific knowledge while safeguarding inference-time data privacy. |
| Outcome: | The proposed model protects private data while enhancing the model's knowledge. |
Knowledge Unlearning for Mitigating Privacy Risks in Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Recent work shows that an adversary can extract training data from Pretrained Language Models including Personally Identifiable Information (PII) such as names, phone numbers, and email addresses. |
| Approach: | They propose to use knowledge unlearning to reduce privacy risks for LMs by performing gradient ascent on target token sequences instead of trying to unlearn all the data at once. |
| Outcome: | The proposed method can give a stronger empirical privacy guarantee in scenarios where the data vulnerable to extraction attacks are known a priori while being much more efficient and robust. |
Can Public Large Language Models Help Private Cross-device Federated Learning? (2024.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have shown that public data can be used to improve privacy-utility trade-offs for large and small language models. |
| Approach: | They propose to use large-scale public data to help differentially private FL training . they propose a distribution matching algorithm with theoretical grounding to sample public data close to private data distribution . |
| Outcome: | The proposed method is efficient and effective for training private models by taking advantage of public data. |
One size does not fit all: Investigating strategies for differentially-private learning across NLP tasks (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing research on the efficiency of differentially-private stochastic gradient descent (DP-SGD) in NLP is inconclusive or even counter-intuitive. |
| Approach: | They propose to use differentially-private stochastic gradient descent (DP-SGD) to preserve privacy in NLP by using modern neural models based on BERT and XtremeDistil architectures to conduct extensive experiments. |
| Outcome: | The proposed models and training strategies provide the best trade-off between privacy and performance on different NLP tasks. |
Differentially Private Learning Needs Better Model Initialization and Self-Distillation (2025.naacl-long)
Copied to clipboard
| Challenge: | Differentially private SGD (DPSGD) enables privacy-preserving training of language models, but often reduces utility, diversity, and linguistic quality. |
| Approach: | They propose a method that initializes a model using data synthesis from a small pre-trained LM with rigorous filtering, applies DP finetuning on private data, and performs self-distillation to refine outputs. |
| Outcome: | The proposed method outperforms vanilla DPSGD with significant improvements in lexical diversity and grammar errors. |
DP-GTR: Differentially Private Prompt Protection via Group Text Rewriting (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for prompt privacy focus on document-level rewriting, neglecting rich, multi-granular representations of text. |
| Approach: | a framework that leverages local differential privacy and composition theorem via group text rewriting is proposed . the framework is compatible with existing rewrite techniques and is publicly available at anonymous.4open.science for reproducibility. |
| Outcome: | DP-GTR is the first framework to integrate document-level and word-level information while exploiting in-context learning to improve privacy and utility. |