Vision Language Model Helps Private Information De-Identification in Vision Data (2025.findings-acl)
Copied to clipboard
| Challenge: | Visual Language Models (VLMs) have gained popularity due to their ability to solve imagerelated tasks. |
| Approach: | They propose a framework to enhance privacy awareness of visual language models . they use a specialized instruction-tuning dataset and a tailored training methodology . |
| Outcome: | The proposed framework outperforms existing approaches in handling private information. |
Similar Papers
Defeating Cerberus: Privacy-Leakage Mitigation in Vision Language Models (2026.findings-eacl)
Copied to clipboard
Boyang Zhang, Istemi Ekin Akkus, Ruichuan Chen, Alice Dethise, Klaus Satzke, Ivica Rimac, Yang Zhang
| Challenge: | Existing models that process multiple modalities of data have been used for multimodal tasks, but their advanced capabilities raise privacy concerns. |
| Approach: | They propose a method to modify the model’s internal states associated with PII-related content and to reduce the risk of PI I leakage by modifying the model's internal state. |
| Outcome: | The proposed method achieves on average 93.3% refusal rate for various PII-related tasks with minimal impact on unrelated model performances. |
Learning and Evaluating a Differentially Private Pre-trained Language Model (2021.findings-emnlp)
Copied to clipboard
Shlomo Hoory, Amir Feder, Avichai Tendler, Sofia Erell, Alon Peled-Cohen, Itay Laish, Hootan Nakhost, Uri Stemmer, Ayelet Benjamini, Avinatan Hassidim, Yossi Matias
| Challenge: | Contextual language models have improved performance but can lead to information leakage . |
| Approach: | They propose a differentially-private word-piece algorithm that allows training a tailored domain-specific vocabulary while maintaining privacy. |
| Outcome: | The proposed model can guarantee privacy while maintaining good model performance. |
PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Models (2024.acl-long)
Copied to clipboard
Haoran Li, Dadi Guo, Donghao Li, Wei Fan, Qi Hu, Xin Liu, Chunkit Chan, Duanyi Yao, Yuan Yao, Yangqiu Song
| Challenge: | generative large language models (LLMs) exhibit surprising capability and integrate previous tasks into a unified text generation formulation. |
| Approach: | They propose a privacy evaluation benchmark to quantify the privacy leakage of language models. |
| Outcome: | The proposed benchmark compares PPLMs with different privacy implementations to find out how privacy leakage is handled. |
A Unified Framework and Dataset for Assessing Societal Bias in Vision-Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have highlighted the existence of social biases within large vision and language models. |
| Approach: | They propose a framework for systematically evaluating gender, race, and age biases in vision-language models with respect to professions. |
| Outcome: | The proposed framework covers all supported inference modes of the recent vision-language models, including image-to-text, text-to image, and image- to-image. |
TextObfuscator: Making Pre-trained Language Model a Privacy Protector via Obfuscating Word Representations (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing inference services are plagued by privacy concerns, such as sharing sensitive data with service providers. |
| Approach: | They propose a framework for protecting inference privacy by applying random perturbations to clustered representations. |
| Outcome: | The proposed framework protects inference privacy by applying random perturbations to clustered representations. |
Large Language Models Can Be Contextual Privacy Protection Learners (2024.emnlp-main)
Copied to clipboard
Yijia Xiao, Yiqiao Jin, Yushi Bai, Yue Wu, Xianjun Yang, Xiao Luo, Wenchao Yu, Xujiang Zhao, Yanchi Liu, Quanquan Gu, Haifeng Chen, Wei Wang, Wei Cheng
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable linguistic comprehension and generation capability, but when applied to specialized industries, they face challenges such as hallucination, insufficient domain knowledge, and failing to incorporate the latest domain knowledge. |
| Approach: | They propose a paradigm for fine-tuning LLMs that effectively injects domain-specific knowledge while safeguarding inference-time data privacy. |
| Outcome: | The proposed model protects private data while enhancing the model's knowledge. |
PII-VisBench: Evaluating Personally Identifiable Information Safety in Vision Language Models Along a Continuum of Visibility (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluations of PII leakage ignore how a subject’s online presence affects privacy alignment. |
| Approach: | They propose a benchmark that evaluates safety through the continuum of online presence by stratifying 200 subjects into four visibility categories: high, medium, low, and zero. |
| Outcome: | The proposed model stratifies 200 subjects into four visibility categories based on the extent and nature of their information available online. |
DPF-CM: A Data Processing Framework with Privacy-Preserving Vector Databases for Chinese Medical LLMs Training and Deployment (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new framework for training data processing for Chinese medical language models is proposed . experimental results show that the framework significantly improves model accuracy . |
| Approach: | They propose a data processing framework for Chinese medical language models training and deployment . the framework is based on a question-oriented model training strategy and privacy preservation . |
| Outcome: | The proposed framework significantly improves model accuracy and reduces privacy leakage by 27%. |
Adversarial Learning of Privacy-Preserving Text Representations for De-Identification of Medical Records (P19-1)
Copied to clipboard
| Challenge: | De-identification is the task of detecting protected health information (PHI) in medical text. |
| Approach: | They propose to create shareable representations of medical text that contain no PHI and can be shared between organizations to create unified datasets for training de-identification models. |
| Outcome: | The proposed representation allows training a simple LSTM-CRF model to an F1 score of 97.4%. |
Robust Utility-Preserving Text Anonymization Based on Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing techniques face challenges of re-identification ability of large language models . anonymizing text that contains sensitive information is crucial for a wide range of applications . |
| Approach: | They propose a framework that integrates three key LLM components to perform anonymization. |
| Outcome: | The proposed model outperforms baselines while maintaining greater data utility in downstream tasks. |