Papers by Niloofar Mireshghallah
Smaller Language Models are Better Zero-shot Machine-Generated Text Detectors (2024.eacl-short)
Copied to clipboard
| Challenge: | Using large language models to detect machine generated text is difficult for humans to distinguish between human-written and machine-generated text. |
| Approach: | They propose to use one language model to detect machine-generated text produced by another language model in a zero-shot way. |
| Outcome: | The proposed methods can detect machine-generated text without additional training/data. |
Privacy Ripple Effects from Adding or Removing Personal Information in Language Model Training (2025.findings-acl)
Copied to clipboard
Jaydeep Borkar, Matthew Jagielski, Katherine Lee, Niloofar Mireshghallah, David A. Smith, Christopher A. Choquette-Choo
| Challenge: | PII is a sensitive information that can be removed from large-language model training due to evolving curation techniques, or because it was recently scraped for retraining. |
| Approach: | They characterize a phenomenon where PII that appeared earlier in training becomes extractable at a later step after fine-tuning on other PI I. |
| Outcome: | The authors show that PII memorization is a dynamic property of a model that evolves throughout training pipelines and depends on commonly altered design choices. |
ALPACA AGAINST VICUNA: Using LLMs to Uncover Memorization of LLMs (2025.naacl-long)
Copied to clipboard
Aly M. Kassem, Omar Mahmoud, Niloofar Mireshghallah, Hyunwoo Kim, Yulia Tsvetkov, Yejin Choi, Sherif Saad, Santu Rana
| Challenge: | Existing studies have shown that pre-trained LLMs emit training data up to 150 more often than in regular operation. |
| Approach: | They propose a black-box prompt optimization method where an attacker LLM agent uncovers higher levels of memorization in a victim agent . |
| Outcome: | The proposed method shows 23.7% more overlap with training data compared to state-of-the-art baselines. |
LatticeGen: Hiding Generated Text in a Lattice for Privacy-Aware Large Language Model Generation on Cloud (2024.findings-naacl)
Copied to clipboard
Mengke Zhang, Tianxing He, Tianle Wang, Lu Mi, Niloofar Mireshghallah, Binyi Chen, Hao Wang, Yulia Tsvetkov
| Challenge: | Currently, the server controls the generated text, but users can't keep it private . prompted generation is a common interaction paradigm for large language models on cloud . |
| Approach: | They propose a protocol where the server handles most of the computation while the client controls the sampling operation. |
| Outcome: | The proposed protocol protects both prompt and generation under strong attacks. |
CopyBench: Measuring Literal and Non-Literal Reproduction of Copyright-Protected Text in Language Model Generation (2024.emnlp-main)
Copied to clipboard
Tong Chen, Akari Asai, Niloofar Mireshghallah, Sewon Min, James Grimmelmann, Yejin Choi, Hannaneh Hajishirzi, Luke Zettlemoyer, Pang Wei Koh
| Challenge: | Existing studies focus on literal copying, but current methods reduce literal copy but not non-literal copying. |
| Approach: | They propose a benchmark to measure literal and non-literal copying in LMs . they use copyrighted fiction books as text sources to assess literal copying . |
| Outcome: | The proposed model measures literal and non-literal copying in copyrighted texts . large models show significantly more copying, with literal copying rates increasing . |
Simple Temporal Adaptation to Changing Label Sets: Hashtag Prediction via Dense KNN (2023.emnlp-main)
Copied to clipboard
Niloofar Mireshghallah, Nikolai Vogler, Junxian He, Omar Florez, Ahmed El-Kishky, Taylor Berg-Kirkpatrick
| Challenge: | Existing methods to adapt to temporal change of user-generated social media data are stale without retraining. |
| Approach: | They propose a non-parametric dense retrieval technique to adapt to temporal change . they use a Twitter dataset to study temporal distribution shift in tweet-hashtag prediction . |
| Outcome: | The proposed method improves over the best static parametric baseline on a year-long Twitter dataset while avoiding costly re-training. |
Information-Guided Identification of Training Data Imprint in (Proprietary) Large Language Models (2025.naacl-long)
Copied to clipboard
Abhilasha Ravichander, Jillian Fisher, Taylor Sorensen, Ximing Lu, Maria Antoniak, Bill Yuchen Lin, Niloofar Mireshghallah, Chandra Bhagavatula, Yejin Choi
| Challenge: | Lack of transparency in training data is limiting external oversight and inspection of LLMs for issues such as copyright infringement and data contamination. |
| Approach: | They propose a method to identify training data known to proprietary LLMs without requiring access to model weights or token probabilities by using information-guided probes. |
| Outcome: | The proposed method can identify training data known to proprietary LLMs without access to model weights or token probabilities. |
Differentially Private Learning Needs Better Model Initialization and Self-Distillation (2025.naacl-long)
Copied to clipboard
| Challenge: | Differentially private SGD (DPSGD) enables privacy-preserving training of language models, but often reduces utility, diversity, and linguistic quality. |
| Approach: | They propose a method that initializes a model using data synthesis from a small pre-trained LM with rigorous filtering, applies DP finetuning on private data, and performs self-distillation to refine outputs. |
| Outcome: | The proposed method outperforms vanilla DPSGD with significant improvements in lexical diversity and grammar errors. |