| Challenge: | Text anonymization is essential for developing and deploying AI in high stakes domains . tools for redacting directly identifying content are unlikely to guarantee 100% recall . |
| Approach: | They propose a method for privacy-preserving synthetic text generation that leverages HIPS theory and de-identification principles. |
| Outcome: | The proposed method achieves a strong balance between privacy protection and utility on legal and clinical datasets. |
Similar Papers
Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe (2023.acl-long)
Copied to clipboard
Xiang Yue, Huseyin Inan, Xuechen Li, Girish Kumar, Julia McAnallen, Hoda Shajari, Huan Sun, David Levitan, Robert Sim
| Challenge: | Privacy concerns have increased in data-driven products due to the tendency of machine learning models to memorize sensitive training data. |
| Approach: | They propose a method for generating useful synthetic text with a formal privacy guarantee by fine-tuning a pretrained generative language model with DP. |
| Outcome: | The proposed method produces synthetic text competitive in terms of utility with its non-private counterpart, while providing strong protection against potential privacy leakages. |
Private prediction for large-scale synthetic text generation (2024.findings-emnlp)
Copied to clipboard
Kareem Amin, Alex Bie, Weiwei Kong, Alexey Kurakin, Natalia Ponomareva, Umar Syed, Andreas Terzis, Sergei Vassilvitskii
| Challenge: | Existing approaches to generate differentially private text using large language models are classified into several categories. |
| Approach: | They propose a private prediction framework that generates differentially private synthetic text using large language models via private prediction. |
| Outcome: | The proposed approach generates high-quality synthetic data points at reasonable privacy levels while protecting the privacy of users who contributed to the dataset. |
Evaluating Differentially Private Synthetic Data Generation in High-Stakes Domains (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a lack of anonymization of sensitive text data hinders development of NLP tools . poorly anonymized sensitive data cannot be easily shared with annotators or external researchers . |
| Approach: | They propose to use synthetic data to generate differentially private language models in place of real data to facilitate NLP development without compromising privacy. |
| Outcome: | The proposed model can be used to train public models without compromising privacy. |
DPGA-TextSyn: Differentially Private Genetic Algorithm for Synthetic Text Generation (2025.findings-acl)
Copied to clipboard
Zhonghao Sun, Zhiliang Tian, Yiping Song, Yuyi Si, Juhua Zhang, Minlie Huang, Kai Lu, Zeyu Xiong, Xinwang Liu, Dongsheng Li
| Challenge: | Existing methods to fine-tune large language models pose privacy risks . researchers have synthesized data with strong generation capabilities closed-source LLMs to alleviate this problem . |
| Approach: | They propose to combine general LLMs with genetic algorithm to produce relevant and diverse synthetic text under differential privacy constraints. |
| Outcome: | The proposed method significantly improves the performance of the model in downstream tasks while ensuring privacy. |
KnowledgeSG: Privacy-Preserving Synthetic Text Generation with Knowledge Distillation from Server (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to train large language models on private data are not effective because they rely on a local model for generation, resulting in a performance decline, or expose private data to API servers. |
| Approach: | They propose a client-server framework which enhances synthetic data quality and improves model performance while ensuring privacy. |
| Outcome: | The proposed framework improves model performance and privacy while learning local knowledge from the private data with differential privacy (DP) and distilling professional knowledge from server. |
Private Synthetic Text Generation with Diffusion Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent research shows diffusion models are capable of generating synthetics texts . but are they also good in generating private data if the training was under differential privacy? |
| Approach: | They examine whether diffusion models are capable of generating synthetics texts . they critically assess previous work on private text generation with diffusion models . |
| Outcome: | The proposed model outperforms auto-regressive models in generating private images despite unmet privacy assumptions . the proposed model is open-source and can be used for other purposes . |
Private Seeds, Public LLMs: Realistic and Privacy-Preserving Synthetic Data Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are a powerful tool for creating synthetic replicas of private text. |
| Approach: | They propose a method for creating privacy-preserving synthetic data using private seeds and a formal differential privacy mechanism. |
| Outcome: | The proposed method achieves high fidelity to private data while providing strong privacy protection. |
CAPE: Context-Aware Private Embeddings for Private Language Learning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to obtain text representations or embeddings with these models encoding personally identifiable information may lead to privacy leaks. |
| Approach: | They propose a novel approach which combines differential privacy and adversarial learning to preserve privacy during training of embeddings. |
| Outcome: | The proposed approach reduces private information leakage by 3% over the current method. |
SynthTextEval: Synthetic Text Data Generation and Evaluation for High-Stakes Domains (2025.emnlp-demos)
Copied to clipboard
Krithika Ramesh, Daniel Smolyak, Zihao Zhao, Nupoor Gandhi, Ritu Agarwal, Margrét V. Bjarnadóttir, Anjalie Field
| Challenge: | SynthTextEval is a toolkit for conducting comprehensive evaluations of synthetic text. |
| Approach: | They propose a toolkit for conducting comprehensive evaluations of synthetic text using large language models. |
| Outcome: | The proposed toolkit can be run over any dataset, but it is aimed at two high-stakes domains: healthcare and law. |
Generating Synthetic Free-text Medical Records with Low Re-identification Risk using Masked Language Modeling (2025.naacl-srw)
Copied to clipboard
| Challenge: | Existing methods to generate medical records using Causal Language Modelling are limited due to privacy concerns. |
| Approach: | They propose a method for generating medical records using Masked Language Modelling using Causal language models. |
| Outcome: | The proposed method produces high-quality synthetic data with a re-identification risk of only 3.5% and a patient recall of 96%. |