Papers by Mukund Srinath
LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing (2024.emnlp-main)
Copied to clipboard
Jiangshu Du, Yibo Wang, Wenting Zhao, Zhongfen Deng, Shuaiqi Liu, Renze Lou, Henry Zou, Pranav Narayanan Venkit, Nan Zhang, Mukund Srinath, Haoran Zhang, Vipul Gupta, Yinghui Li, Tao Li, Fei Wang, Qin Liu, Tianlin Liu, Pengzhi Gao, Congying Xia, Chen Xing, Cheng Jiayang, Zhaowei Wang, Ying Su, Raj Shah, Ruohao Guo, Jing Gu, Haoran Li, Kangda Wei, Zihao Wang, Lu Cheng, Surangika Ranathunga, Meng Fang, Jie Fu, Fei Liu, Ruihong Huang, Eduardo Blanco, Yixin Cao, Rui Zhang, Philip Yu, Wenpeng Yin
| Challenge: | a comparative analysis of paper (meta-)reviews by large language models (LLMs) aims to identify and distinguish LLMs from human activities . |
| Approach: | They present a comparative analysis to identify and distinguish LLM activities from human activities. |
| Outcome: | The proposed analysis aims to improve recognition of instances when someone implicitly uses LLMs for reviewing activities. |
An Audit on the Perspectives and Challenges of Hallucinations in NLP (2024.emnlp-main)
Copied to clipboard
Pranav Narayanan Venkit, Tatiana Chakravorti, Vipul Gupta, Heidi Biggs, Mukund Srinath, Koustava Goswami, Sarah Rajtmajer, Shomir Wilson
| Challenge: | 103 peer-reviewed publications on hallucination in large language models (LLMs) are characterized by a lack of agreement with the term ‘hallucination’ in the field of NLP. |
| Approach: | They examine 103 peer-reviewed publications on hallucination in large language models (LLMs) and conduct a survey with 171 practitioners from the field of NLP and AI to capture varying perspectives on halllucination. |
| Outcome: | The findings highlight the need for explicit definitions and frameworks outlining hallucination within NLP and highlight potential challenges. |
A Study of Implicit Bias in Pretrained Language Models against People with Disabilities (2022.coling-1)
Copied to clipboard
| Challenge: | Pretrained language models exhibit sociodemographic biases, such as against gender and race, raising concerns of downstream biase in language technologies. |
| Approach: | They propose to use word embedding-based and transformer-based PLMs to test for the presence of biases against people with disabilities (PWDs) |
| Outcome: | The proposed models favor ableist language, despite their sociodemographic biases against race and gender. |
Privacy at Scale: Introducing the PrivaSeer Corpus of Web Privacy Policies (2021.acl-long)
Copied to clipboard
| Challenge: | Existing tools to interpret privacy policies have been used to understand them but there is a lack of large privacy policy corpora to simplify the process. |
| Approach: | They propose to use a corpus of 1,005,380 English language privacy policies collected from the web to create semi-supervised and unsupervised models to interpret and simplify privacy policies. |
| Outcome: | The proposed model outperforms all other publicly available privacy policy corpora and is ten times larger than the next largest public collection of privacy policies combined. |
PseudoSeer: a Search Engine for Pseudocode (2026.findings-acl)
Copied to clipboard
| Challenge: | PseudoSeer is a search engine for academic pseudocode that indexes over 320,000 implementations extracted from 2.2 million arXiv papers. |
| Approach: | They propose to use caption-reference pairs to match short queries with a median length of five words against long documents composed primarily of natural language with limited LaTeX notation. |
| Outcome: | The proposed algorithm outperforms the best pretrained model by 8.7 points and achieves 66.5% R@10 . |
Automated Detection and Analysis of Data Practices Using A Real-World Corpus (2024.findings-acl)
Copied to clipboard
| Challenge: | a crowd-sourced annotation tool matches data practices with policy excerpts . the complexity of privacy policies often deter users from reading them . |
| Approach: | They propose an automated approach to identify and visualize data practices within privacy policies at different levels of detail. |
| Outcome: | The proposed approach matches data practices with policy excerpts at different levels of detail. |
The Sentiment Problem: A Critical Survey towards Deconstructing Sentiment Analysis (2023.emnlp-main)
Copied to clipboard
Pranav Venkit, Mukund Srinath, Sanjana Gautam, Saranya Venkatraman, Vipul Gupta, Rebecca Passonneau, Shomir Wilson
| Challenge: | Existing research reveals a notable absence of interdisciplinary endeavors to comprehend the social dimensions of sentiment analysis, encompassing aspects like emotion and fairness. |
| Approach: | They propose an ethics sheet encompassing critical inquiries to guide practitioners in ensuring equitable utilization of SA. |
| Outcome: | The proposed ethics sheet outlines the importance of adopting an interdisciplinary approach to defining sentiment in SA and offers a pragmatic solution for its implementation. |