Differentially Private Next-Token Prediction of Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are becoming increasingly important for ensuring privacy, but DP-SGD overestimates an adversary’s capabilities in having white box access. |
| Approach: | They propose a private prediction protocol for next-token prediction that utilizes the inherent stochasticity of next-toning and a public model to achieve Differential Privacy. |
| Outcome: | The proposed protocol outperforms DP-SGD and DP training methods for privacy on large datasets. |
Similar Papers
Private prediction for large-scale synthetic text generation (2024.findings-emnlp)
Copied to clipboard
Kareem Amin, Alex Bie, Weiwei Kong, Alexey Kurakin, Natalia Ponomareva, Umar Syed, Andreas Terzis, Sergei Vassilvitskii
| Challenge: | Existing approaches to generate differentially private text using large language models are classified into several categories. |
| Approach: | They propose a private prediction framework that generates differentially private synthetic text using large language models via private prediction. |
| Outcome: | The proposed approach generates high-quality synthetic data points at reasonable privacy levels while protecting the privacy of users who contributed to the dataset. |
Differentially Private Language Models for Secure Data Sharing (2022.emnlp-main)
Copied to clipboard
| Challenge: | a variety of deanonymization attacks allow the re-identification of individuals from tabular data. |
| Approach: | They propose to train a language model in a differentially private manner and sample data from it . they find that the model generates fluent textual datasets with privacy guarantees . |
| Outcome: | The proposed methods outperform direct classifiers with DP-SGD in the real-world. |
Differentially Private Learning Needs Better Model Initialization and Self-Distillation (2025.naacl-long)
Copied to clipboard
| Challenge: | Differentially private SGD (DPSGD) enables privacy-preserving training of language models, but often reduces utility, diversity, and linguistic quality. |
| Approach: | They propose a method that initializes a model using data synthesis from a small pre-trained LM with rigorous filtering, applies DP finetuning on private data, and performs self-distillation to refine outputs. |
| Outcome: | The proposed method outperforms vanilla DPSGD with significant improvements in lexical diversity and grammar errors. |
Differentially Private Knowledge Distillation via Synthetic Text Generation (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language models (LLMs) are achieving state-of-the-art performance in many downstream tasks, but data privacy is a major challenge for practitioners. |
| Approach: | They propose a differentially private knowledge distillation algorithm that exploits the knowledge of a teacher LLM and a student's output distribution. |
| Outcome: | The proposed algorithm significantly improves the utility over baselines on the Big Patent dataset, with strong privacy parameters, =2. |
Model-based Large Language Model Customization as Service (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing large language model services require users to upload data for fine-tuning . current methods for customization are noisy and require sensitive domain data . |
| Approach: | *Llamdex is a framework that facilitates LLM customization as a service . client uploads pre-trained domain-specific *models* rather than data . |
| Outcome: | *Llamdex* framework improves domain-specific accuracy by up to 26% over state-of-the-art private data synthesis methods . |
Large-Scale Differentially Private BERT (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that scaling up the batch size to millions improves the utility of a DP-SGD step for BERT. |
| Approach: | They propose to use differentially private SGD to pretrain BERT-Large with a batch size of millions to improve the utility of the DP-SGD step. |
| Outcome: | The proposed approach achieves a masked language model accuracy of 60.5% at a batch size of 2M, which is a reasonable privacy setting. |
Can Public Large Language Models Help Private Cross-device Federated Learning? (2024.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have shown that public data can be used to improve privacy-utility trade-offs for large and small language models. |
| Approach: | They propose to use large-scale public data to help differentially private FL training . they propose a distribution matching algorithm with theoretical grounding to sample public data close to private data distribution . |
| Outcome: | The proposed method is efficient and effective for training private models by taking advantage of public data. |
DPGA-TextSyn: Differentially Private Genetic Algorithm for Synthetic Text Generation (2025.findings-acl)
Copied to clipboard
Zhonghao Sun, Zhiliang Tian, Yiping Song, Yuyi Si, Juhua Zhang, Minlie Huang, Kai Lu, Zeyu Xiong, Xinwang Liu, Dongsheng Li
| Challenge: | Existing methods to fine-tune large language models pose privacy risks . researchers have synthesized data with strong generation capabilities closed-source LLMs to alleviate this problem . |
| Approach: | They propose to combine general LLMs with genetic algorithm to produce relevant and diverse synthetic text under differential privacy constraints. |
| Outcome: | The proposed method significantly improves the performance of the model in downstream tasks while ensuring privacy. |
TextFusion: Privacy-Preserving Pre-trained Model Inference via Token Fusion (2022.emnlp-main)
Copied to clipboard
Xin Zhou, Jinzhu Lu, Tao Gui, Ruotian Ma, Zichu Fei, Yuran Wang, Yong Ding, Yibo Cheung, Qi Zhang, Xuanjing Huang
| Challenge: | Existing methods to preserve inference privacy are available as cloud services . however, the risk of privacy leakage remains, according to recent studies . |
| Approach: | They propose a method to preserve inference privacy by fusing token representations in the cloud. |
| Outcome: | The proposed method preserves inference privacy without sacrificing performance on different scenarios. |
One size does not fit all: Investigating strategies for differentially-private learning across NLP tasks (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing research on the efficiency of differentially-private stochastic gradient descent (DP-SGD) in NLP is inconclusive or even counter-intuitive. |
| Approach: | They propose to use differentially-private stochastic gradient descent (DP-SGD) to preserve privacy in NLP by using modern neural models based on BERT and XtremeDistil architectures to conduct extensive experiments. |
| Outcome: | The proposed models and training strategies provide the best trade-off between privacy and performance on different NLP tasks. |