Papers by Priyaranjan Pattnayak
IndicJR: A Judge-Free Benchmark of Jailbreak Robustness in South Asian Languages (2026.eacl-industry)
Copied to clipboard
| Challenge: | Indic Jailbreak Robustness (IJR) is a judge-free benchmark for adversarial safety across 12 languages. |
| Approach: | They propose a judge-free benchmark for adversarial safety across 12 languages . they find contracts inflate refusals but do not stop jailbreaks . |
| Outcome: | The proposed benchmarks cover 45,216 prompts in JSON and Free tracks. |
Hard Negative Mining for Domain-Specific Retrieval in Enterprise Systems (2025.acl-industry)
Copied to clipboard
| Challenge: | Existing methods for lexical retrieval struggle due to semantic mismatches and overlapping terminologies, and ambiguous abbreviations common in specialized fields like finance and cloud computing. |
| Approach: | They propose a scalable hard-negative mining framework that dynamically selects semantically challenging but contextually irrelevant documents to enhance deployed re-ranking models. |
| Outcome: | The proposed framework improves on public domain datasets and shows that it is generalizable and ready for real-world applications. |
Crowdsource, Crawl, or Generate? Creating SEA-VL, a Multicultural Vision-Language Dataset for Southeast Asia (2025.acl-long)
Copied to clipboard
Samuel Cahyawijaya, Holy Lovenia, Joel Ruben Antony Moniz, Tack Hwa Wong, Mohammad Rifqi Farhansyah, Thant Thiri Maung, Frederikus Hudi, David Anugraha, Muhammad Ravi Shulthan Habibi, Muhammad Reza Qorib, Amit Agarwal, Joseph Marvin Imperial, Hitesh Laxmichand Patel, Vicky Feliren, Bahrul Ilmi Nasution, Manuel Antonio Rufino, Genta Indra Winata, Rian Adam Rajagede, Carlos Rafael Catalan, Mohamed Fazli Mohamed Imam, Priyaranjan Pattnayak, Salsabila Zahirah Pranida, Kevin Pratama, Yeshil Bangera, Adisai Na-Thalang, Patricia Nicole Monderin, Yueqi Song, Christian Simon, Lynnette Hui Xian Ng, Richardy Lobo Sapan, Taki Hasan Rafi, Bin Wang, null Supryadi, Kanyakorn Veerakanjana, Piyalitt Ittichaiwong, Matthew Theodore Roque, Karissa Vincentio, Takdanai Kreangphet, Phakphum Artkaew, Kadek Hendrawan Palgunadi, Yanzhi Yu, Rochana Prih Hastuti, William Nixon, Mithil Bangera, Adrian Xuan Wei Lim, Aye Hninn Khine, Hanif Muhammad Zhafran, Teddy Ferdinan, Audra Aurora Izzani, Ayushman Singh, Evan Evan, Jauza Akbar Krito, Michael Anugraha, Fenal Ashokbhai Ilasariya, Haochen Li, John Amadeo Daniswara, Filbert Aurelian Tjiaranata, Eryawan Presma Yulianrifat, Can Udomcharoenchaikit, Fadil Risdian Ansori, Mahardika Krisna Ihsani, Giang Nguyen, Anab Maulana Barik, Dan John Velasco, Rifo Ahmad Genadi, Saptarshi Saha, Chengwei Wei, Isaiah Edri W. Flores, Kenneth Chen Ko Han, Anjela Gail D. Santos, Wan Shen Lim, Kaung Si Phyo, Tim Santos, Meisyarah Dwiastuti, Jiayun Luo, Jan Christian Blaise Cruz, Ming Shan Hee, Ikhlasul Akmal Hanif, M.Alif Al Hakim, Muhammad Rizky Sya’ban, Kun Kerdthaisong, Lester James Validad Miranda, Fajri Koto, Tirana Noor Fatyanosa, Alham Fikri Aji, Jostin Jerico Rosal, Jun Kevin, Robert Wijaya, Onno P. Kampman, Ruochen Zhang, Börje F. Karlsson, Peerat Limkonchotiwat
| Challenge: | Southeast Asia is underrepresented in vision-language research . SEA-VL is an open-source initiative dedicated to developing culturally relevant datasets for SEA languages. |
| Approach: | They propose to use crowdsourced, automated image crawling and synthetic image generation to develop culturally relevant datasets for SEA languages. |
| Outcome: | The proposed datasets capture SEA cultural nuances and contexts better than existing datasets. |
MVTamperBench: Evaluating Robustness of Vision-Language Models (2025.findings-acl)
Copied to clipboard
Amit Agarwal, Srikant Panda, Angeline Charles, Hitesh Laxmichand Patel, Bhargava Kumar, Priyaranjan Pattnayak, Taki Hasan Rafi, Tejaswini Kumar, Hansa Meghwani, Karan Gupta, Dong-Kyu Chae
| Challenge: | Multimodal Large Language Models (MLLMs) have been a key advance in video understanding but their vulnerability to adversarial tampering remains underexplored. |
| Approach: | They evaluate MLLMs against five prevalent tampering techniques to assess their robustness . they use a tampered video format to examine the vulnerability of ML models . |
| Outcome: | The benchmark evaluates MLLMs against five prevalent tampering techniques based on 19 video manipulation tasks. |
SweEval: Do LLMs Really Swear? A Safety Benchmark for Testing Limits for Enterprise Use (2025.naacl-industry)
Copied to clipboard
Hitesh Laxmichand Patel, Amit Agarwal, Arion Das, Bhargava Kumar, Srikant Panda, Priyaranjan Pattnayak, Taki Hasan Rafi, Tejaswini Kumar, Dong-Kyu Chae
| Challenge: | Large Language Models (LLMs) are increasingly being used for communication tasks across different regions. |
| Approach: | They propose a benchmark to evaluate whether Large Language Models are ethically aligned and can be used in real-world situations. |
| Outcome: | The proposed benchmark evaluates whether LLMs comply with or resist swearing instructions and assesses their alignment with ethical frameworks, cultural nuances, and language comprehension capabilities. |
ReproEvalCard: A Reporting Standard for Reproducible Evaluation of LLM Pipelines (2026.acl-short)
Copied to clipboard
| Challenge: | Existing evaluation standards for multistage pipelines are inconsistent, leaving the reproducibility and independent validation of published evaluations unclear. |
| Approach: | They propose a lightweight reporting standard that specifies the minimum artifacts required to reproduce and validate LLM evaluations. |
| Outcome: | The proposed standard audits 55 pipeline-based LLM papers published between 2022 and 2025 and quantifies the availability of reproducibility-critical evaluation artifacts. |