Runhui Wang, Yefan Tao, Adit Krishnan, Luyang Kong, Xuanqing Liu, Yuqian Deng, Yunzhao Yang, Henrik Johnson, Andrew Borthwick, Shobhit Gupta, Aditi Gundlapalli, Davor Golac
| Challenge: | Data deduplication is a critical task in data management and mining, focused on consolidating duplicate records that refer to the same entity. |
| Approach: | They propose to use a dataset with 1,000,000 unlabeled synthetic PII profiles and a subset of 10,000 pairs curated and labeled as matches or non-matches. |
| Outcome: | The proposed datasets contain synthetic profiles built from publicly available sources that do not represent real individuals. |
Similar Papers
De-Identification of Sensitive Personal Data in Datasets Derived from IIT-CDIP (2024.emnlp-main)
Copied to clipboard
Stefan Larson, Nicole Lima, Santiago Diaz, Amogh Joshi, Siddharth Betala, Jamiu Suleiman, Yash Mathur, Kaushal Prajapati, Ramla Alakraa, Junjie Shen, Temi Okotore, Kevin Leach
| Challenge: | Large volumes of data are becoming increasingly important for training machine learning models for document understanding tasks like classification, information extraction, and visual question answering. |
| Approach: | They propose a data de-identification pipeline that replaces sensitive data with synthetic, but realistic, data that preserves the utility of de-identified documents. |
| Outcome: | The proposed method preserves the utility of the de-identified documents so that they can continue to be used in various document understanding applications. |
CAPID: Context-Aware PII Detection for Question-Answering Systems (2026.eacl-srw)
Copied to clipboard
Mariia Ponomarenko, Sepideh Abedini, Masoumeh Shafieinejad, D. B. Emerson, Shubhankar Mohapatra, Xi He
| Challenge: | Existing approaches mainly redact all PII, disregarding the fact that some may be contextually relevant to the user’s question, resulting in a degradation of response quality. |
| Approach: | They propose a method that fine-tunes a locally owned small language model that filters sensitive information before it is passed to LLMs for QA. |
| Outcome: | The proposed approach outperforms baselines in span, relevance and type accuracy while preserving significantly higher utility under anonymization. |
SPY: Enhancing Privacy with Synthetic PII Detection Dataset (2025.naacl-srw)
Copied to clipboard
| Challenge: | Historically, Named Entity Recognition (NER) has been employed for PII detection, but PI I entities constitute a subset of NER entities. |
| Approach: | They propose to use Large Language Models to generate a synthetic dataset that emulates real-world PII scenarios and validate its quality. |
| Outcome: | The proposed dataset is validated and provides a benchmark for PII detection. |
K-LegalDeID: A Benchmark Dataset and KLUEBERT-CRF for De-identification in Korean Court Judgments (2026.eacl-long)
Copied to clipboard
| Challenge: | Korean legal system mandates public access to court judgments to ensure judicial transparency, but this requirement conflicts with privacy protection obligations due to the prevalence of Personally Identifiable Information (PII) in legal documents. |
| Approach: | They propose a large-scale benchmark dataset and an efficient KLUEBERT-CRF model for de-identification for Korean court judgments. |
| Outcome: | The proposed model achieves state-of-the-art performance with an entity-level micro F1 score of 0.9923. |
PII-Bench: Evaluating Query-Aware Privacy Protection Systems (2026.acl-long)
Copied to clipboard
| Challenge: | Existing models do not detect PII in user prompts, despite their convenience . current models show significant limitations in determining PI I query relevance . |
| Approach: | They propose a query-unrelated PII masking strategy and propose PIi-Bench . they propose 'quick-and-easy' PI I masking with a user query and context description . |
| Outcome: | The proposed model performs well in basic PII detection, but shows significant limitations in query relevance. |
PersonaBench: Evaluating AI Models on Understanding Personal Information through Accessing (Synthetic) Private User Data (2025.findings-acl)
Copied to clipboard
Juntao Tan, Liangwei Yang, Zuxin Liu, Zhiwei Liu, Rithesh R N, Tulika Manoj Awalgaonkar, Jianguo Zhang, Weiran Yao, Ming Zhu, Shirley Kokane, Silvio Savarese, Huan Wang, Caiming Xiong, Shelby Heinecke
| Challenge: | Existing research lacks direct access to such data, making benchmarking difficult due to privacy concerns. |
| Approach: | They propose a synthetic data pipeline that generates realistic user profiles and private documents and a benchmark to evaluate models' ability to understand personal information. |
| Outcome: | The proposed pipeline generates realistic user profiles and private documents, enabling PersonaBench, a benchmark for evaluating models’ ability to understand personal information. |
PII-VisBench: Evaluating Personally Identifiable Information Safety in Vision Language Models Along a Continuum of Visibility (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluations of PII leakage ignore how a subject’s online presence affects privacy alignment. |
| Approach: | They propose a benchmark that evaluates safety through the continuum of online presence by stratifying 200 subjects into four visibility categories: high, medium, low, and zero. |
| Outcome: | The proposed model stratifies 200 subjects into four visibility categories based on the extent and nature of their information available online. |
Easy, Reproducible and Quality-Controlled Data Collection with CROWDAQ (2020.emnlp-demos)
Copied to clipboard
Qiang Ning, Hao Wu, Pradeep Dasigi, Dheeru Dua, Matt Gardner, Robert L. Logan IV, Ana Marasović, Zhen Nie
| Challenge: | Efficient data collection is important for advancing research and building time-sensitive applications. |
| Approach: | They propose an open-source platform that standardizes the data collection pipeline . it includes customizable user interface components, automated annotator qualification, and saved pipelines . |
| Outcome: | The proposed platform simplifies data annotation significantly on diverse datasets . it can be used by researchers and engineers to improve reproducibility and minimize overhead . |
Adversarial Learning of Privacy-Preserving Text Representations for De-Identification of Medical Records (P19-1)
Copied to clipboard
| Challenge: | De-identification is the task of detecting protected health information (PHI) in medical text. |
| Approach: | They propose to create shareable representations of medical text that contain no PHI and can be shared between organizations to create unified datasets for training de-identification models. |
| Outcome: | The proposed representation allows training a simple LSTM-CRF model to an F1 score of 97.4%. |
PAPILLON: Privacy Preservation from Internet-based and Local Language Model Ensembles (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing research has studied privacy in LLM training data memorization, but it does not prevent users from disclosing PII at inference time. |
| Approach: | They propose a task for chaining API-based and local LLMs that uses public data to construct a benchmark that contains personally identifiable information (PII) |
| Outcome: | The proposed model maintains high response quality for 85.5% of user queries while restricting privacy leakage to only 7.5%. |