KnowledgeSG: Privacy-Preserving Synthetic Text Generation with Knowledge Distillation from Server (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to train large language models on private data are not effective because they rely on a local model for generation, resulting in a performance decline, or expose private data to API servers. |
| Approach: | They propose a client-server framework which enhances synthetic data quality and improves model performance while ensuring privacy. |
| Outcome: | The proposed framework improves model performance and privacy while learning local knowledge from the private data with differential privacy (DP) and distilling professional knowledge from server. |
Similar Papers
Differentially Private Knowledge Distillation via Synthetic Text Generation (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language models (LLMs) are achieving state-of-the-art performance in many downstream tasks, but data privacy is a major challenge for practitioners. |
| Approach: | They propose a differentially private knowledge distillation algorithm that exploits the knowledge of a teacher LLM and a student's output distribution. |
| Outcome: | The proposed algorithm significantly improves the utility over baselines on the Big Patent dataset, with strong privacy parameters, =2. |
Private prediction for large-scale synthetic text generation (2024.findings-emnlp)
Copied to clipboard
Kareem Amin, Alex Bie, Weiwei Kong, Alexey Kurakin, Natalia Ponomareva, Umar Syed, Andreas Terzis, Sergei Vassilvitskii
| Challenge: | Existing approaches to generate differentially private text using large language models are classified into several categories. |
| Approach: | They propose a private prediction framework that generates differentially private synthetic text using large language models via private prediction. |
| Outcome: | The proposed approach generates high-quality synthetic data points at reasonable privacy levels while protecting the privacy of users who contributed to the dataset. |
Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe (2023.acl-long)
Copied to clipboard
Xiang Yue, Huseyin Inan, Xuechen Li, Girish Kumar, Julia McAnallen, Hoda Shajari, Huan Sun, David Levitan, Robert Sim
| Challenge: | Privacy concerns have increased in data-driven products due to the tendency of machine learning models to memorize sensitive training data. |
| Approach: | They propose a method for generating useful synthetic text with a formal privacy guarantee by fine-tuning a pretrained generative language model with DP. |
| Outcome: | The proposed method produces synthetic text competitive in terms of utility with its non-private counterpart, while providing strong protection against potential privacy leakages. |
Private Seeds, Public LLMs: Realistic and Privacy-Preserving Synthetic Data Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are a powerful tool for creating synthetic replicas of private text. |
| Approach: | They propose a method for creating privacy-preserving synthetic data using private seeds and a formal differential privacy mechanism. |
| Outcome: | The proposed method achieves high fidelity to private data while providing strong privacy protection. |
Differentially Private Language Models for Secure Data Sharing (2022.emnlp-main)
Copied to clipboard
| Challenge: | a variety of deanonymization attacks allow the re-identification of individuals from tabular data. |
| Approach: | They propose to train a language model in a differentially private manner and sample data from it . they find that the model generates fluent textual datasets with privacy guarantees . |
| Outcome: | The proposed methods outperform direct classifiers with DP-SGD in the real-world. |
Resource-Efficient Anonymization of Textual Data via Knowledge Distillation from Large Language Models (2025.coling-industry)
Copied to clipboard
| Challenge: | Existing approaches to anonymize textual data from large language models pose privacy risks due to their API-based access. |
| Approach: | They propose a method to distill large language models into smaller encoder-only models via named entity recognition coupled with regular expressions to create a lightweight model capable of effective anonymization. |
| Outcome: | The proposed approach reduces computational overhead while maintaining semantic integrity of data. |
DPGA-TextSyn: Differentially Private Genetic Algorithm for Synthetic Text Generation (2025.findings-acl)
Copied to clipboard
Zhonghao Sun, Zhiliang Tian, Yiping Song, Yuyi Si, Juhua Zhang, Minlie Huang, Kai Lu, Zeyu Xiong, Xinwang Liu, Dongsheng Li
| Challenge: | Existing methods to fine-tune large language models pose privacy risks . researchers have synthesized data with strong generation capabilities closed-source LLMs to alleviate this problem . |
| Approach: | They propose to combine general LLMs with genetic algorithm to produce relevant and diverse synthetic text under differential privacy constraints. |
| Outcome: | The proposed method significantly improves the performance of the model in downstream tasks while ensuring privacy. |
Private Synthetic Text Generation with Diffusion Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent research shows diffusion models are capable of generating synthetics texts . but are they also good in generating private data if the training was under differential privacy? |
| Approach: | They examine whether diffusion models are capable of generating synthetics texts . they critically assess previous work on private text generation with diffusion models . |
| Outcome: | The proposed model outperforms auto-regressive models in generating private images despite unmet privacy assumptions . the proposed model is open-source and can be used for other purposes . |
Evaluating Differentially Private Synthetic Data Generation in High-Stakes Domains (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a lack of anonymization of sensitive text data hinders development of NLP tools . poorly anonymized sensitive data cannot be easily shared with annotators or external researchers . |
| Approach: | They propose to use synthetic data to generate differentially private language models in place of real data to facilitate NLP development without compromising privacy. |
| Outcome: | The proposed model can be used to train public models without compromising privacy. |
Differentially Private Synthetic Text Generation for Retrieval-Augmented Generation (RAG) (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing private RAG methods rely on query-time differential privacy (DP) Existing studies have identified significant privacy risks when their databases contain sensitive information. |
| Approach: | They propose a framework that generates differentially private RAG databases using LLMs . Unlike prior methods, the synthetic text can be reused once created . |
| Outcome: | Experiments show that DP-SynRAG achieves superior performance to state-of-the-art RAG systems while maintaining a fixed privacy budget. |