Papers by Reza Shokri
Context-Aware Membership Inference Attacks against Pre-trained Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Prior Membership Inference Attacks on pre-trained Large Language Models fail at LLMs due to ignoring the generative nature of LLM data. |
| Approach: | They propose a method that adapts MIA statistical tests to the perplexity dynamics of subsequences within a data point. |
| Outcome: | The proposed method significantly outperforms prior approaches, revealing context-dependent memorization patterns in pre-trained LLMs. |
Smaller Language Models are Better Zero-shot Machine-Generated Text Detectors (2024.eacl-short)
Copied to clipboard
| Challenge: | Using large language models to detect machine generated text is difficult for humans to distinguish between human-written and machine-generated text. |
| Approach: | They propose to use one language model to detect machine-generated text produced by another language model in a zero-shot way. |
| Outcome: | The proposed methods can detect machine-generated text without additional training/data. |
Watermark Smoothing Attacks against Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Watermarking is a key technique for detecting AI-generated text. |
| Approach: | They propose a method to selectively smooth watermarks by leveraging the relationship between the model’s confidence and detectability. |
| Outcome: | The proposed method selectively smoothes watermark traces while preserving text quality. |
Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks (2022.emnlp-main)
Copied to clipboard
| Challenge: | Prior attempts at measuring leakage of MLMs via membership inference attacks have been inconclusive, implying potential robustness of Mlms to privacy attacks. |
| Approach: | They propose a stronger membership inference attack based on likelihood ratio hypothesis testing that involves an additional MLM to more accurately quantify the privacy risks of memorization in MLMs. |
| Outcome: | The proposed attack improves the AUC of prior membership inference attacks from 0.66 to an alarmingly high 0.90 level on models trained on medical notes. |
Rethinking Hallucinations: Correctness, Consistency, and Prompt Multiplicity (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing hallucination evaluations focus only on correctness and often overlook consistency . a significant inconsistency in benchmarks like Med-HALT suggests hallucianation-related harms have been misunderstood. |
| Approach: | They propose a framework for quantifying consistency in hallucination evaluations . they find that detection techniques detect consistency, not correctness . |
| Outcome: | The proposed framework uncovers critical limitations in hallucination evaluations. |