Privacy-Preserving Graph Convolutional Networks for Text Classification (2022.lrec-1)
Copied to clipboard
| Challenge: | Graph convolutional networks (GCNs) are powerful for representation learning on documents that naturally occur as graphs, but sensitive personal information is prone to privacy leaks. |
| Approach: | They propose a method that adapts differentially-private gradient-based training to GCNs and conduct experiments using two optimizers on five NLP datasets in two languages. |
| Outcome: | The proposed method improves baseline privacy bounds by 2.7 while retaining competitive F1 scores while providing strong privacy guarantees. |
Similar Papers
How reparametrization trick broke differentially-private text representation learning (2022.acl-short)
Copied to clipboard
| Challenge: | Differential privacy (DP) is a formal mathematical treatment of privacy protection . it guarantees how much privacy can be lost in the worst case . adapting DP mechanisms to NLP properly is largely non-trivial task . |
| Approach: | They propose to use differential privacy to learn text representations using DPText to quantify and guarantee how much privacy can be lost in the worst case. |
| Outcome: | The proposed methods are falsely claimed to be differentially private and violate privacy loss guarantees. |
One size does not fit all: Investigating strategies for differentially-private learning across NLP tasks (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing research on the efficiency of differentially-private stochastic gradient descent (DP-SGD) in NLP is inconclusive or even counter-intuitive. |
| Approach: | They propose to use differentially-private stochastic gradient descent (DP-SGD) to preserve privacy in NLP by using modern neural models based on BERT and XtremeDistil architectures to conduct extensive experiments. |
| Outcome: | The proposed models and training strategies provide the best trade-off between privacy and performance on different NLP tasks. |
Differentially Private Natural Language Models: Recent Advances and Future Directions (2024.findings-eacl)
Copied to clipboard
| Challenge: | Recent advances in deep learning have led to great success in various natural language processing tasks. |
| Approach: | They propose a systematic review of recent advances in DP deep learning models . they discuss some differences and additional challenges of DP-NLP . |
| Outcome: | The proposed method can prevent reconstruction attacks and protect against potential side knowledge while maintaining the privacy of sensitive data. |
Differentially Private Representation for NLP: Formal Guarantee and An Empirical Study on Privacy and Fairness (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to learn text representations can encode private information of the input, thus can be exploited to recover such information with reasonable accuracy. |
| Approach: | They propose a novel approach to preserve privacy of the extracted representation from text by combining differential privacy with dropout. |
| Outcome: | The proposed approach preserves privacy of the extracted representation from text while masking words via dropout can enhance privacy. |
Privacy-preserving Neural Representations of Text (D18-1)
Copied to clipboard
| Challenge: | a specific type of attack is used to characterize the privacy of neural representations for NLP tasks, in the context of privacy protection. |
| Approach: | They propose several defense methods based on modified training objectives and characterize the tradeoff between privacy and the utility of neural representations. |
| Outcome: | The proposed defenses improve the privacy of neural representations and characterize the tradeoff between privacy and utility of representations. |
Privacy Regularization: Joint Privacy-Utility Optimization in LanguageModels (2021.naacl-main)
Copied to clipboard
Fatemehsadat Mireshghallah, Huseyin Inan, Marcello Hasegawa, Victor Rühle, Taylor Berg-Kirkpatrick, Robert Sim
| Challenge: | Neural language models have a high capacity for memorization of training samples . however, this can cause privacy degradation and disparate impact on subgroups of users . |
| Approach: | They propose two privacy-preserving regularization methods for training language models that enable joint optimization of utility and privacy. |
| Outcome: | The proposed methods have favorable utility-privacy trade-off, faster training and uniform treatment of under-represented subgroups. |
ADePT: Auto-encoder based Differentially Private Text Transformation (2021.eacl-main)
Copied to clipboard
| Challenge: | Differential privacy is an important privacy concern when building statistical models on data containing sensitive information. |
| Approach: | They propose a utility-preserving differentially private text transformation algorithm using auto-encoders that can be used to transform text to offer robustness against attacks and produce transformations with high semantic quality. |
| Outcome: | The proposed model performs better against membership inference attacks while offering lower to no degradation in the utility of the underlying transformation process compared to baselines. |
Privacy-Preserving Natural Language Processing (2023.eacl-tutorials)
Copied to clipboard
| Challenge: | This tutorial will help the NLP community to get familiar with current research in privacy-preserving methods. |
| Approach: | This tutorial will help the NLP community to get familiar with current research in privacy-preserving methods. |
| Outcome: | The tutorial will cover membership inference, differential privacy, homomorphic encryption, or federated learning, all with typical use-cases and potential pitfalls. |
Selective Differential Privacy for Language Modeling (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to protect sensitive data from leaking are over-pessimistic and undifferentiated. |
| Approach: | They propose a new privacy notion, selective differential privacy, to provide rigorous privacy guarantees on the sensitive portion of the data to improve model utility. |
| Outcome: | The proposed privacy-preserving mechanism achieves better utility while remaining safe under various privacy attacks compared to baselines. |
Synthetic Query Generation for Privacy-Preserving Deep Retrieval Systems using Differentially Private Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | DP-training methods that require per-example gradients are difficult to implement . however, a method that prioritizes query privacy is not feasible. |
| Approach: | They propose a method that prioritizes ensuring query privacy prior to training a deep retrieval system. |
| Outcome: | The proposed method shows that it improves retrieval quality compared to direct DP-training while maintaining query-level privacy guarantees. |