Papers by Prannaya Gupta
Evaluating AI for Finance: Is AI Credible at Assessing Investment Risk Appetite? (2025.emnlp-industry)
Copied to clipboard
Divij Chawla, Ashita Bhutada, Duc Anh Do, Abhinav Raghunathan, Vinod Sp, Cathy Guo, Dar Win Liew, Prannaya Gupta, Rishabh Bhardwaj, Rajat Bhardwaj, Soujanya Poria
| Challenge: | Our analysis was conducted on proprietary systems and open-weight models . FINRISKEVAL analyzed 1,720 profiles spanning a broad spectrum of possible risk categories . |
| Approach: | They evaluated proprietary AI systems and open-weight models to assess investment risk appetite using carefully curated user profiles. |
| Outcome: | The proposed models exhibit significant variance when user attributes that should not influence risk computation are changed. |
WalledEval: A Comprehensive Safety Evaluation Toolkit for Large Language Models (2024.emnlp-demo)
Copied to clipboard
Prannaya Gupta, Le Yau, Hao Low, I-Shiang Lee, Hugo Lim, Yu Teoh, Koh Hng, Dar Liew, Rishabh Bhardwaj, Rajat Bhardwaj, Soujanya Poria
| Challenge: | Potential harms include training data leakage, biases in responses and decision-making, and unauthorized use for purposes such as terrorism and the generation of sexually explicit content. |
| Approach: | WalledEval is a comprehensive AI safety testing toolkit designed to evaluate large language models. |
| Outcome: | The framework supports both LLM and judge benchmarking and incorporates custom mutators to test safety against various text-style mutations such as future tense and paraphrasing. |