Papers by Natalia Ponomareva
Training Text-to-Text Transformers with Privacy Guarantees (2022.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in NLP often stem from large transformer-based pre-trained models. |
| Approach: | They propose differentially private (DP) training as a potential mitigation for models that can memorize parts of training data. |
| Outcome: | The proposed model can memorize parts of training data and mitigate memorization concerns. |
Synthetic Query Generation for Privacy-Preserving Deep Retrieval Systems using Differentially Private Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | DP-training methods that require per-example gradients are difficult to implement . however, a method that prioritizes query privacy is not feasible. |
| Approach: | They propose a method that prioritizes ensuring query privacy prior to training a deep retrieval system. |
| Outcome: | The proposed method shows that it improves retrieval quality compared to direct DP-training while maintaining query-level privacy guarantees. |
Private prediction for large-scale synthetic text generation (2024.findings-emnlp)
Copied to clipboard
Kareem Amin, Alex Bie, Weiwei Kong, Alexey Kurakin, Natalia Ponomareva, Umar Syed, Andreas Terzis, Sergei Vassilvitskii
| Challenge: | Existing approaches to generate differentially private text using large language models are classified into several categories. |
| Approach: | They propose a private prediction framework that generates differentially private synthetic text using large language models via private prediction. |
| Outcome: | The proposed approach generates high-quality synthetic data points at reasonable privacy levels while protecting the privacy of users who contributed to the dataset. |