Anonymity at Risk? Assessing Re-Identification Capabilities of Large Language Models in Court Decisions (2024.findings-naacl)
Copied to clipboard
| Challenge: | Despite high re-identification rates on Wikipedia, even the best LLMs struggled with court decisions. |
| Approach: | They construct an anonymized Wikipedia dataset to investigate re-identification risks . they also introduce new metrics to measure performance . |
| Outcome: | The proposed model can be used to identify individuals in court decisions, but it fails in the vast majority of cases. |
Similar Papers
Robust Utility-Preserving Text Anonymization Based on Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing techniques face challenges of re-identification ability of large language models . anonymizing text that contains sensitive information is crucial for a wide range of applications . |
| Approach: | They propose a framework that integrates three key LLM components to perform anonymization. |
| Outcome: | The proposed model outperforms baselines while maintaining greater data utility in downstream tasks. |
Combating Security and Privacy Issues in the Era of Large Language Models (2024.naacl-tutorials)
Copied to clipboard
| Challenge: | a tutorial aims to provide a summary of risks and vulnerabilities in large language models . a number of studies have focused on security, privacy and copyright aspects of LLMs . |
| Approach: | This tutorial seeks to provide a systematic summary of risks and vulnerabilities in large language models . authors will discuss security, privacy and copyright aspects of LLMs . |
| Outcome: | This tutorial aims to provide a systematic summary of risks and vulnerabilities in large language models . it will also outline emerging challenges in security, privacy and reliability of LLMs . |
Anonymisation Models for Text Data: State of the art, Challenges and Future Directions (2021.acl-long)
Copied to clipboard
| Challenge: | a paper examines the problem of automated text anonymisation . text anonymization is a prerequisite for secure sharing of documents containing sensitive information about individuals. |
| Approach: | They propose to incorporate explicit measures of disclosure risk into the text anonymisation process to reduce the risk of errors. |
| Outcome: | The proposed approach is based on a case study in which the authors outline the benefits and limitations of the proposed methods. |
Learning to Refuse: Towards Mitigating Privacy Risks in LLMs (2025.coling-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit remarkable capabilities in understanding and generating natural languages, but can inadvertently memorize private information, posing significant privacy risks. |
| Approach: | They propose to use a dataset to evaluate machine unlearning methods for protecting personal data in a realistic scenario. |
| Outcome: | The proposed model outperforms baseline methods by 5.65 points and protects target individuals’ personal data while maintaining general capabilities. |
De-Anonymization at Scale via Tournament-Style Attribution (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are rapidly gaining widespread adoption in real-world use . authors propose a method for attributing authorship among tens of thousands of candidate texts . |
| Approach: | They propose a large-language-model-based method for attributing authorship among tens of thousands of candidate texts. |
| Outcome: | The proposed method improves accuracy and ranking precision over previous approaches. |
Humans or LLMs as the Judge? A Study on Judgement Bias (2024.emnlp-main)
Copied to clipboard
| Challenge: | Proprietary models such as GPT-4, Claude, Gemini-Pro and others are being democratized to improve evaluations of LLMs. |
| Approach: | They propose a framework that is free from referencing groundtruth annotations for investigating **Misinformation Oversight Bias**, **Gender Bia**,**Authority Bia* and **Beauty Bia's** on LLM and human judges. |
| Outcome: | The proposed framework investigates **Misinformation Oversight Bias**, **Gender Bia**,**Authority Bia* and **Beauty Bia' on LLM and human judges. |
From Citations to Criticality: Predicting Legal Decision Influence in the Multilingual Swiss Jurisprudence (2025.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to evaluating the importance of legal cases are manual and resource-intensive. |
| Approach: | They propose a dataset that uses two-tier labels to evaluate case criticality . they use the LD-Label to identify cases published as Leading Decisions and the Citation-L Label to rank cases by their citation frequency and recency. |
| Outcome: | The Criticality Prediction dataset outperforms existing approaches to evaluate case criticality . the proposed model outperformed the existing models in a zero-shot setting . |
LLMs and Copyright Risks: Benchmarks and Mitigation Approaches (2025.naacl-tutorial)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have revolutionized natural language processing, but their widespread use has raised significant copyright concerns. |
| Approach: | This tutorial will provide an overview of relevant copyright principles and their application to AI and examine specific copyright issues in LLM development and deployment. |
| Outcome: | The course will provide an overview of relevant copyright principles and their application to AI, followed by an examination of specific copyright issues in LLM development and deployment. |
SecureSQL: Evaluating Data Leakage of Large Language Models as Natural Language Interfaces to Databases (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on the vulnerability of large language models to SQL injection have been limited. |
| Approach: | They propose to evaluate the potential of language models to leak sensitive data when generating SQL queries. |
| Outcome: | The proposed model with the best performance has an accuracy of 61.7%, compared to humans who achieve 94% accuracy. |
LLMs are Privacy Erasable (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new study examines the privacy of large language models and their capabilities . the study aims to address the balance between the convenience of LLMs and user privacy concerns . |
| Approach: | They propose a strategy that safeguards user prompt while accessing LLM cloud services . they evaluate the efficacy of their method across prominent LLM benchmarks . |
| Outcome: | The proposed method thwarts reconstruction attacks and improves model performance . it also surpasses the results reported in official model cards . |