Papers by Ninareh Mehrabi
Data Advisor: Dynamic Data Curation for Safety Alignment of Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have explored using LLMs for efficient data collection. |
| Approach: | They propose a method that takes into account the characteristics of the desired dataset and monitors the status of the generated data. |
| Outcome: | The proposed method improves safety and quality of three representative large language models against safety issues without sacrificing model utility. |
Diagnosing Memorization in Chain-of-Thought Reasoning, One Token at a Time (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) perform well on reasoning benchmarks but often fail when inputs alter slightly, raising concerns about overreliance on memorization. |
| Approach: | They propose a framework for Source-aware Token-level Identification of Memorization which attributes each token in a reasoning chain to one of multiple memorization sources based on their statistical co-occurrence with the token in the pretraining corpus. |
| Outcome: | The proposed framework attributes each token in a reasoning chain to one of multiple memorization sources based on their statistical co-occurrence with the token in the pretraining corpus. |
MICo: Preventative Detoxification of Large Language Models through Inhibition Control (2024.findings-naacl)
Copied to clipboard
Roy Siegelmann, Ninareh Mehrabi, Palash Goyal, Prasoon Goyal, Lisa Bauer, Jwala Dhamala, Aram Galstyan, Rahul Gupta, Reza Ghanadan
| Challenge: | Large Language Models (LLMs) have a tendency to devolve into toxic degeneration . model may classify prompts as toxic or non-toxic and categorically refuse to respond to those deemed toxic. |
| Approach: | They propose a mechanism for LLM detoxification by labeling acceptable and unacceptable examples and including a corresponding acceptable rewrite with every unacceptable example. |
| Outcome: | The proposed model improves on the baseline model and shows that it detects and rewrites toxic and harmful examples. |
DiCoRe: Enhancing Zero-shot Event Detection via Divergent-Convergent LLM Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Understanding the complex event ontology, extracting domain-specific triggers from the passage, and structuring them appropriately overloads and limits the utility of Large Language Models (LLMs). |
| Approach: | They propose a divergent-convergent reasoning framework that decouples the task of ED using Dreamer and Grounder. |
| Outcome: | The proposed framework outperforms baselines on six datasets across five domains and nine LLMs, achieving 4–7% average gains over the best baseline. |
On Localizing and Deleting Toxic Memories in Large Language Models (2025.findings-naacl)
Copied to clipboard
Anubrata Das, Manoj Kumar, Ninareh Mehrabi, Anil Ramakrishna, Anna Rumshisky, Kai-Wei Chang, Aram Galstyan, Morteza Ziyadi, Rahul Gupta
| Challenge: | Existing methods to reduce toxic generation in large language models are not fully understood. |
| Approach: | They propose to understand the mechanisms that drive toxic generation in large language models by using memory localization to reduce toxic generation. |
| Outcome: | The proposed method reduces toxic generation from 62.86% to 28.61%, but it also improves generation quality. |
Robust Conversational Agents against Imperceptible Toxicity Triggers (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing work to generate adversarial attacks is costly and not scalable . despite the abundance of research in this area, little attention has been given to adversarials . |
| Approach: | They propose an adversarial attack mechanism that mitigates toxic language generation . they propose a defense mechanism that is scalable and can be generalized . |
| Outcome: | The proposed defense is effective at avoiding toxic language generation even against imperceptible toxicity triggers while preserving conversational flow. |
The steerability of large language models toward data-driven personas (2024.naacl-long)
Copied to clipboard
Junyi Li, Charith Peris, Ninareh Mehrabi, Palash Goyal, Kai-Wei Chang, Aram Galstyan, Richard Zemel, Rahul Gupta
| Challenge: | Large language models generate biased responses where opinions of certain groups and populations are underrepresented. |
| Approach: | They propose a data-driven notion of persona that allows for a more nuanced understanding of different (latent) social groups present in the population. |
| Outcome: | The proposed method improves model steerability by 57% over baselines. |
Attribute Controlled Fine-tuning for Large Language Models: A Case Study on Detoxification (2024.findings-emnlp)
Copied to clipboard
Tao Meng, Ninareh Mehrabi, Palash Goyal, Anil Ramakrishna, Aram Galstyan, Richard Zemel, Kai-Wei Chang, Rahul Gupta, Charith Peris
| Challenge: | Using a sequence-level constraint, we regularize the LLMtraining by penalizing the KL divergence between the desired output distribution and the LRM’s posterior. |
| Approach: | They propose a constraint learning schema forfine-tuning Large Language Models with attribute control by penalizing the KL divergence be-tween the desired output distribution and the LLM's posterior. |
| Outcome: | The proposed approach improves the performance of large language models while enhancing their utility and generation quality. |
Lawyers are Dishonest? Quantifying Representational Harms in Commonsense Knowledge Resources (2021.emnlp-main)
Copied to clipboard
| Challenge: | Commonsense knowledge bases are mostly human-generated and reflect societal biases . a filtering-based approach can reduce the issues in both resources and models but leads to a performance drop . |
| Approach: | They propose a filtering-based approach to mitigating representational harms in ConceptNet and GenericsKB . they propose filtered-based approaches can reduce issues in both resources and models but leads to performance drop . |
| Outcome: | The proposed approach reduces issues in resources and models but leads to performance drop . the paper proposes a filtering-based approach that reduces biases but leaves room for future work . |
FLIRT: Feedback Loop In-context Red Teaming (2024.emnlp-main)
Copied to clipboard
Ninareh Mehrabi, Palash Goyal, Christophe Dupuy, Qian Hu, Shalini Ghosh, Richard Zemel, Kai-Wei Chang, Aram Galstyan, Rahul Gupta
| Challenge: | Recent work has evaluated the vulnerabilities of large generative models, such as DALL-E, ChatGPT, and GPT-4. |
| Approach: | They propose an automatic red teaming framework that evaluates a given black-box model and exposes its vulnerabilities against unsafe and inappropriate content generation. |
| Outcome: | The proposed framework evaluates a given black-box model and exposes its vulnerabilities against unsafe and inappropriate content generation. |
Tokenization Matters: Navigating Data-Scarce Tokenization for Gender Inclusive Language Technologies (2024.findings-naacl)
Copied to clipboard
Anaelia Ovalle, Ninareh Mehrabi, Palash Goyal, Jwala Dhamala, Kai-Wei Chang, Richard Zemel, Aram Galstyan, Yuval Pinter, Rahul Gupta
| Challenge: | a recent study documented the harmful limitations of gender binary-centric large language models . data scarcity is a known culprit, but the precise mechanisms through which scarcity affects this behavior remain underexplored. |
| Approach: | They propose to use BPE tokenization to enforce consistent tokenization across gendered pronouns to improve neopronoun proficiency. |
| Outcome: | The proposed methods outperform finetuning with standard BPE, and improve neopronoun proficiency. |
Prompt Perturbation Consistency Learning for Robust Language Models (2024.findings-eacl)
Copied to clipboard
Yao Qiang, Subhrangshu Nandi, Ninareh Mehrabi, Greg Ver Steeg, Anoop Kumar, Anna Rumshisky, Aram Galstyan
| Challenge: | Large language models have demonstrated impressive performance on a number of natural language processing tasks, such as question answering and text summarization. |
| Approach: | They propose a method to reduce the performance drop of large language models by regularizing the divergence between losses from clean and perturbed samples. |
| Outcome: | The proposed approach recovers on average 59% and 69% of the performance drop for IC and SF tasks while using ten times fewer augmented data samples. |
Resolving Ambiguities in Text-to-Image Generative Models (2023.acl-long)
Copied to clipboard
Ninareh Mehrabi, Palash Goyal, Apurv Verma, Jwala Dhamala, Varun Kumar, Qian Hu, Kai-Wei Chang, Richard Zemel, Aram Galstyan, Rahul Gupta
| Challenge: | ambiguities can lead to misinterpretation and miscommunication in natural language . resolving ambiguity is notoriously hard for machines . |
| Approach: | They propose a framework to disambiguate prompts given to generative models by soliciting clarifications from the end user. |
| Outcome: | The proposed framework generates more faithful images better aligned with user intention in the presence of ambiguities. |
Towards Safety Reasoning in LLMs: AI-agentic Deliberation for Policy-embedded CoT Data Creation (2025.findings-acl)
Copied to clipboard
Tharindu Kumarage, Ninareh Mehrabi, Anil Ramakrishna, Xinyan Zhao, Richard Zemel, Kai-Wei Chang, Aram Galstyan, Rahul Gupta, Charith Peris
| Challenge: | Safety reasoning paradigms require high-quality policy-embedded chain-of-thought datasets . generating such data through human annotations is prohibitively expensive . |
| Approach: | They propose AIDSAFE: Agentic Iterative Deliberation for Safety Reasoning . AIDS AFE leverages multi-agent deliberation to iteratively expand reasoning on safety policies . |
| Outcome: | The proposed model improves policy adherence and reasoning quality while maintaining acceptable utility and over-refusal accuracy. |
SWAN: Semantic Watermarking with Abstract Meaning Representation (2026.acl-long)
Copied to clipboard
Ziping Ye, Gourab Dey, Christos Christodoulopoulos, Charith Peris, Anil Ramakrishna, Weitong Ruan, Aram Galstyan, Kai-Wei Chang, Rahul Gupta, Ninareh Mehrabi
| Challenge: | Existing methods to embed signatures by adjusting token selection preferences during text generation are highly sensitive to paraphrasing and synonyms. |
| Approach: | They propose a framework that embeds watermark signatures into the semantic structure of a sentence using Abstract Meaning Representation (AMR). |
| Outcome: | Empirical evaluation shows SWAN matches state-of-the-art detection performance on unaltered watermarked text while improving robustness against paraphrasing. |
Tree-of-Traversals: A Zero-Shot Reasoning Algorithm for Augmenting Black-box Language Models with Knowledge Graphs (2024.acl-long)
Copied to clipboard
Elan Markowitz, Anil Ramakrishna, Jwala Dhamala, Ninareh Mehrabi, Charith Peris, Rahul Gupta, Kai-Wei Chang, Aram Galstyan
| Challenge: | Knowledge graphs (KGs) complement Large Language Models (LLMs) by providing reliable, structured, domain-specific, and up-to-date external knowledge. |
| Approach: | They propose a zero-shot reasoning algorithm that augments black-box LLMs with one or more KGs. |
| Outcome: | The proposed algorithm significantly improves performance on question answering and KG question answering tasks. |