Papers by Rishabh Maheshwary
Adversarial Examples for Evaluating Math Word Problem Solvers (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing MWP solvers do not understand language and its relation with numbers, and their accuracy is unclear. |
| Approach: | They propose two methods to generate adversarial attacks to evaluate the robustness of existing MWP solvers. |
| Outcome: | The proposed method reduces the accuracy of existing MWP solvers by over 40% on two benchmark datasets. |
A Strong Baseline for Query Efficient Attacks in a Black Box Setting (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing black box search methods are inefficient as they do not consider the amount of queries required to generate adversarial attacks. |
| Approach: | They propose a query efficient attack strategy to generate plausible adversarial examples on text classification and entailment tasks. |
| Outcome: | The proposed attack reduces query count by 75% across all datasets and target models compared to prior attacks in a limited query setting. |
Prompting with Phonemes: Enhancing LLMs’ Multilinguality for Non-Latin Script Languages (2025.naacl-long)
Copied to clipboard
Hoang H Nguyen, Khyati Mahajan, Vikas Yadav, Julian Salazar, Philip S. Yu, Masoud Hashemi, Rishabh Maheshwary
| Challenge: | Multilingual LLMs have achieved remarkable benchmark performance, but continue to underperform on non-Latin script languages. |
| Approach: | They propose to integrate phonemic transcriptions as complementary signals to induce script-invariant representations by integrating phonemic and orthographic transcriptions. |
| Outcome: | The proposed approach improves performance for Latin and non-Latin script languages, with 12.6% performance improvement and 15.1% performance improvement compared to randomized ICL retrieval. |
M2Lingual: Enhancing Multilingual, Multi-Turn Instruction Alignment in Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches to collect instruction fine-tuning data are limited due to their toxicity, privacy and toxicity concerns. |
| Approach: | They propose to use a two-step taxonomy to transform a small set of human written instructions into complex and challenging conversations. |
| Outcome: | M2Lingual has 175K conversations across 70 languages with a balanced mix of high, low and mid-resourced languages. |
Variable Layerwise Quantization: A Simple and Effective Approach to Quantize LLMs (2025.findings-acl)
Copied to clipboard
Razvan-Gabriel Dumitru, Vikas Yadav, Rishabh Maheshwary, Paul Ioan Clotan, Sathwik Tejaswi Madhusudhan, Mihai Surdeanu
| Challenge: | a meta quantization approach quantizes different layers of a large language model at different bit levels. |
| Approach: | They propose a meta quantization approach that quantizes different layers of a large language model at different bit levels. |
| Outcome: | The proposed method quantizes the most important layers to higher bit precision and less important layers at lower bits. |
M-RewardBench: Evaluating Reward Models in Multilingual Settings (2025.acl-long)
Copied to clipboard
Srishti Gureja, Lester James Validad Miranda, Shayekh Bin Islam, Rishabh Maheshwary, Drishti Sharma, Gusti Triandi Winata, Nathan Lambert, Sebastian Ruder, Sara Hooker, Marzieh Fadaee
| Challenge: | Reward models (RMs) are primarily trained and evaluated in English and their capabilities in multilingual settings remain understudied. |
| Approach: | They construct a multilingual RM evaluation benchmark that tests the chat, safety, reasoning, and translation capabilities of RMs in 23 languages. |
| Outcome: | The proposed model performs better for high-resource languages and improves with translation quality. |
Enhancing Alignment using Curriculum Learning & Ranked Preferences (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Direct Preference Optimization (DPO) is an effective technique that leverages pairwise preference data to align LLMs to human preferences. |
| Approach: | They propose to use pairwise preference data to create multiple preference pairs for a given prompt. |
| Outcome: | The proposed method outperforms standard DPO on MTbench, Vicuna bench, and WizardLM with a score of 7.43 on the test sets. |
Practice Makes a Solver Perfect: Data Augmentation for Math Word Problem Solvers (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing Math Word Problem solvers do not generalize well and rely on superficial cues to achieve high performance. |
| Approach: | They propose several data augmentation techniques to increase the size of existing MWP datasets by five folds by deploying them to a benchmark dataset. |
| Outcome: | The proposed methods increase the generalization and robustness of existing solvers by over five percentage points on benchmark datasets. |