Papers by Palash Goyal
Data Advisor: Dynamic Data Curation for Safety Alignment of Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have explored using LLMs for efficient data collection. |
| Approach: | They propose a method that takes into account the characteristics of the desired dataset and monitors the status of the generated data. |
| Outcome: | The proposed method improves safety and quality of three representative large language models against safety issues without sacrificing model utility. |
MICo: Preventative Detoxification of Large Language Models through Inhibition Control (2024.findings-naacl)
Copied to clipboard
Roy Siegelmann, Ninareh Mehrabi, Palash Goyal, Prasoon Goyal, Lisa Bauer, Jwala Dhamala, Aram Galstyan, Rahul Gupta, Reza Ghanadan
| Challenge: | Large Language Models (LLMs) have a tendency to devolve into toxic degeneration . model may classify prompts as toxic or non-toxic and categorically refuse to respond to those deemed toxic. |
| Approach: | They propose a mechanism for LLM detoxification by labeling acceptable and unacceptable examples and including a corresponding acceptable rewrite with every unacceptable example. |
| Outcome: | The proposed model improves on the baseline model and shows that it detects and rewrites toxic and harmful examples. |
PlanGEN: A Multi-Agent Framework for Generating Planning and Reasoning Trajectories for Complex Problem Solving (2025.emnlp-main)
Copied to clipboard
Mihir Parmar, Xin Liu, Palash Goyal, Yanfei Chen, Long Le, Swaroop Mishra, Hossein Mobahi, Jindong Gu, Zifeng Wang, Hootan Nakhost, Chitta Baral, Chen-Yu Lee, Tomas Pfister, Hamid Palangi
| Challenge: | Existing methods for natural planning lack constraint-guided iterative verification and adaptive selection . a recent study found that LLMs are not good at such planning. |
| Approach: | They propose a model-agnostic and easily scalable agent framework with three key components: constraint, verification, and selection agents. |
| Outcome: | The proposed framework improves inference-time algorithms on NATURAL PLAN and OlympiadBench benchmarks. |
Faithful Model Evaluation for Model-Based Metrics (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies do not consider variance change due to metric model errors, which can lead to wrong conclusions. |
| Approach: | They establish the mathematical foundation of significance testing for model-based metrics . they show that metric errors can change the conclusions in certain experiments . |
| Outcome: | The proposed method can be used to derive accurate conclusions using model evaluations. |
The steerability of large language models toward data-driven personas (2024.naacl-long)
Copied to clipboard
Junyi Li, Charith Peris, Ninareh Mehrabi, Palash Goyal, Kai-Wei Chang, Aram Galstyan, Richard Zemel, Rahul Gupta
| Challenge: | Large language models generate biased responses where opinions of certain groups and populations are underrepresented. |
| Approach: | They propose a data-driven notion of persona that allows for a more nuanced understanding of different (latent) social groups present in the population. |
| Outcome: | The proposed method improves model steerability by 57% over baselines. |
PLAN-TUNING: Post-Training Language Models to Learn Step-by-Step Planning for Complex Problem Solving (2025.emnlp-main)
Copied to clipboard
Mihir Parmar, Palash Goyal, Xin Liu, Yiwen Song, Mingyang Ling, Chitta Baral, Hamid Palangi, Tomas Pfister
| Challenge: | Recent studies have shown that decomposing complex problems into simple subtasks has significantly boosted the performance of large language models (LLMs). |
| Approach: | They propose a unified post-training framework that distills synthetic task decompositions and fine-tunes smaller LLMs via supervised and reinforcement-learning objectives to improve complex reasoning. |
| Outcome: | The proposed framework outperforms strong baselines on GSM8k and MATH benchmarks and shows that it can improve generalization capabilities on out-of-domain datasets. |
Attribute Controlled Fine-tuning for Large Language Models: A Case Study on Detoxification (2024.findings-emnlp)
Copied to clipboard
Tao Meng, Ninareh Mehrabi, Palash Goyal, Anil Ramakrishna, Aram Galstyan, Richard Zemel, Kai-Wei Chang, Rahul Gupta, Charith Peris
| Challenge: | Using a sequence-level constraint, we regularize the LLMtraining by penalizing the KL divergence between the desired output distribution and the LRM’s posterior. |
| Approach: | They propose a constraint learning schema forfine-tuning Large Language Models with attribute control by penalizing the KL divergence be-tween the desired output distribution and the LLM's posterior. |
| Outcome: | The proposed approach improves the performance of large language models while enhancing their utility and generation quality. |
FLIRT: Feedback Loop In-context Red Teaming (2024.emnlp-main)
Copied to clipboard
Ninareh Mehrabi, Palash Goyal, Christophe Dupuy, Qian Hu, Shalini Ghosh, Richard Zemel, Kai-Wei Chang, Aram Galstyan, Rahul Gupta
| Challenge: | Recent work has evaluated the vulnerabilities of large generative models, such as DALL-E, ChatGPT, and GPT-4. |
| Approach: | They propose an automatic red teaming framework that evaluates a given black-box model and exposes its vulnerabilities against unsafe and inappropriate content generation. |
| Outcome: | The proposed framework evaluates a given black-box model and exposes its vulnerabilities against unsafe and inappropriate content generation. |
Tokenization Matters: Navigating Data-Scarce Tokenization for Gender Inclusive Language Technologies (2024.findings-naacl)
Copied to clipboard
Anaelia Ovalle, Ninareh Mehrabi, Palash Goyal, Jwala Dhamala, Kai-Wei Chang, Richard Zemel, Aram Galstyan, Yuval Pinter, Rahul Gupta
| Challenge: | a recent study documented the harmful limitations of gender binary-centric large language models . data scarcity is a known culprit, but the precise mechanisms through which scarcity affects this behavior remain underexplored. |
| Approach: | They propose to use BPE tokenization to enforce consistent tokenization across gendered pronouns to improve neopronoun proficiency. |
| Outcome: | The proposed methods outperform finetuning with standard BPE, and improve neopronoun proficiency. |
Resolving Ambiguities in Text-to-Image Generative Models (2023.acl-long)
Copied to clipboard
Ninareh Mehrabi, Palash Goyal, Apurv Verma, Jwala Dhamala, Varun Kumar, Qian Hu, Kai-Wei Chang, Richard Zemel, Aram Galstyan, Rahul Gupta
| Challenge: | ambiguities can lead to misinterpretation and miscommunication in natural language . resolving ambiguity is notoriously hard for machines . |
| Approach: | They propose a framework to disambiguate prompts given to generative models by soliciting clarifications from the end user. |
| Outcome: | The proposed framework generates more faithful images better aligned with user intention in the presence of ambiguities. |