Efficiently and Thoroughly Anonymizing a Transformer Language Model for Dutch Electronic Health Records: a Two-Step Method (2022.lrec-1)
Copied to clipboard
| Challenge: | Neural Networks (NNs) are used to model large amounts of data, such as text data, and have shown to be very useful for language modelling. |
| Approach: | They propose to use a Dutch language model for hospital notes to anonymize a model trained on large amounts of data and publish it online. |
| Outcome: | The proposed method predicts a name-like token 0.2% of the time, compared to the original training data. |
Similar Papers
Adapting Language Models When Training on Privacy-Transformed Data (2022.lrec-1)
Copied to clipboard
| Challenge: | Using data sanitization methods to remove personal information from spoken messages is not effective because privacy-transformed data is unlikely to match the test distribution. |
| Approach: | They propose to use a data sanitization approach to remove personal information from spoken messages by replacing named entities with other words from the same class. |
| Outcome: | The proposed approach removes personal information from the spoken messages using an automatic named entity recognition method. |
Bootstrapping Text Anonymization Models with Distant Supervision (2022.lrec-1)
Copied to clipboard
| Challenge: | Personal data is ubiquitous in text documents. |
| Approach: | They propose a method to bootstrap text anonymization models based on distant supervision by using a knowledge graph to annotate text documents including personal data. |
| Outcome: | The proposed method uses a knowledge graph to annotate text documents including personal data about a subset of individuals. |
Data-Constrained Synthesis of Training Data for De-Identification (2025.acl-long)
Copied to clipboard
| Challenge: | sensitive domains lack widely available datasets due to privacy risks . recent studies have focused on evaluating the privacy of the synthetic text . |
| Approach: | They domain-adapt LLMs to clinical domain and generate synthetic clinical texts . they then generate NER models that can be annotated with tags for PII . |
| Outcome: | The proposed model performs better than the original model using smaller datasets. |
Generating Synthetic Free-text Medical Records with Low Re-identification Risk using Masked Language Modeling (2025.naacl-srw)
Copied to clipboard
| Challenge: | Existing methods to generate medical records using Causal Language Modelling are limited due to privacy concerns. |
| Approach: | They propose a method for generating medical records using Masked Language Modelling using Causal language models. |
| Outcome: | The proposed method produces high-quality synthetic data with a re-identification risk of only 3.5% and a patient recall of 96%. |
TAG: Gradient Attack on Transformer-based Language Models (2021.findings-emnlp)
Copied to clipboard
Jieren Deng, Yijue Wang, Ji Li, Chenghong Wang, Chao Shang, Hang Liu, Sanguthevar Rajasekaran, Caiwen Ding
| Challenge: | Recent studies show that publicly shared gradients in the training process can reveal the private training data to a third-party. |
| Approach: | They propose a gradient attack algorithm to reconstruct the local training data using GLUE benchmarks. |
| Outcome: | The proposed algorithm achieves 1.5x recover rate and 2.5x ROUGE-2 over previous methods without the need of ground truth label. |
DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have demonstrated that pretrained language models memorize and regurgitate a significant portion of training data, including atypical data points that appear only once in the training data. |
| Approach: | They propose a method to locate and erase risky neurons in order to eliminate the impact of privacy data in the model in batches. |
| Outcome: | The proposed method eliminates the impact of privacy data in the model in batches without affecting the model's performance. |
Robust Utility-Preserving Text Anonymization Based on Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing techniques face challenges of re-identification ability of large language models . anonymizing text that contains sensitive information is crucial for a wide range of applications . |
| Approach: | They propose a framework that integrates three key LLM components to perform anonymization. |
| Outcome: | The proposed model outperforms baselines while maintaining greater data utility in downstream tasks. |
Sensitive Data Detection and Classification in Spanish Clinical Text: Experiments with BERT (2020.lrec-1)
Copied to clipboard
| Challenge: | Massive digital data processing can endanger personal data privacy . anonymisation involves removing or replacing sensitive information from data . |
| Approach: | They propose to use a BERT-based sequence labelling model to conduct an experiment on clinical datasets in Spanish. |
| Outcome: | The proposed model outperforms existing models on clinical datasets in Spanish and shows that it is highly competitive with other models. |
Adversarial Learning of Privacy-Preserving Text Representations for De-Identification of Medical Records (P19-1)
Copied to clipboard
| Challenge: | De-identification is the task of detecting protected health information (PHI) in medical text. |
| Approach: | They propose to create shareable representations of medical text that contain no PHI and can be shared between organizations to create unified datasets for training de-identification models. |
| Outcome: | The proposed representation allows training a simple LSTM-CRF model to an F1 score of 97.4%. |
Why Generate When You Can Discriminate? A Novel Technique for Text Classification using Language Models (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for text classification using autoregressive language models are limited . authors propose a novel technique for text classification using autoreregressives . |
| Approach: | They propose a two-step technique for text classification using autoregressive language models . they use a set of perplexity and log-likelihood based numeric features to elicit a text instance . |
| Outcome: | The proposed technique eliminates parameter updates in LMs and does not limit training examples . it is evaluated across 5 datasets and compares with multiple competent baselines . |