Papers by Swaroop Mishra
In-BoXBART: Get Instructions into Biomedical Multi-Task Learning (2022.findings-naacl)
Copied to clipboard
| Challenge: | Experimental results show that the proposed model outperforms single-task baseline by 3% and multi-task (without instruction) baseline by 18% on an average. |
| Approach: | They propose a unified model that can learn all 32 instruction tasks of the BoX without any task-specific modules. |
| Outcome: | The proposed model outperforms single-task baseline by 3% and multi-task (without instruction) baseline by 18% on an average. |
Is a Question Decomposition Unit All We Need? (2022.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LMs) have achieved state-of-the-art performance on many NLP benchmarks. |
| Approach: | They propose to decompose a hard question into simpler questions that are easier for models to answer. |
| Outcome: | The proposed approach significantly improves model performance (24% for GPT3 and 29% for RoBERTa-SQuAD along with a symbolic calculator) by decomposing a hard question into simpler questions that are easier for models to answer. |
Less is More: Summary of Long Instructions is Better for Program Synthesis (2022.emnlp-main)
Copied to clipboard
| Challenge: | Despite the success of large pre-trained language models, they show below-par performance on the larger and more complicated programming related questions. |
| Approach: | They propose to use a human-generated summary of programming questions to improve LMs' performance. |
| Outcome: | The proposed approach outperforms baseline by 8.13% on the APPS dataset and 11.88% on the CodeContests dataset on an average in terms of strict accuracy. |
How Many Data Samples is an Additional Instruction Worth? (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent introduced instruction-paradigm empowers non-expert users to leverage NLP resources by defining a new task in natural language. |
| Approach: | They propose to define a task in natural language without creating task-specific datasets or building models. |
| Outcome: | The proposed model outperforms multitask learning models but is far from state-of-the-art task-specific models. |
Reverse Thinking Makes LLMs Stronger Reasoners (2025.naacl-long)
Copied to clipboard
Justin Chen, Zifeng Wang, Hamid Palangi, Rujun Han, Sayna Ebrahimi, Long Le, Vincent Perot, Swaroop Mishra, Mohit Bansal, Chen-Yu Lee, Tomas Pfister
| Challenge: | Reverse-Enhanced Thinking (RevThink) is a framework for large language models to perform reverse thinking. |
| Approach: | They propose a framework for enhancing forward-backward reasoning by collecting data from a teacher model and employing three objectives to train a student model in a multi-task learning fashion. |
| Outcome: | The proposed framework outperforms a fine-tuning method trained on 10x more forward reasoning on 12 datasets covering commonsense, math, and logical reasoning. |
ILDAE: Instance-Level Difficulty Analysis of Evaluation Data (2022.acl-long)
Copied to clipboard
| Challenge: | Instance-level difficulty analysis of evaluation data is a new field of research that focuses on leveraging instance difficulty in natural language processing. |
| Approach: | They conduct Instance-Level Difficulty Analysis of Evaluation data in a large-scale setup of 23 datasets and demonstrate its five novel applications. |
| Outcome: | The proposed model improves efficiency and accuracy, improves quality and improves Out-of-Domain performance. |
Don’t Blame the Annotator: Bias Already Starts in the Annotation Instructions (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent studies have shown that data collected through crowdsourcing often exhibit various biases that lead to overestimation of model performance. |
| Approach: | They propose to model instruction bias in 14 recent NLU benchmarks by analyzing crowdsourcing instructions and analyzing their results. |
| Outcome: | The proposed model can be over-represented in datasets with a large number of examples, and the results are consistent with previous studies. |
PlanGEN: A Multi-Agent Framework for Generating Planning and Reasoning Trajectories for Complex Problem Solving (2025.emnlp-main)
Copied to clipboard
Mihir Parmar, Xin Liu, Palash Goyal, Yanfei Chen, Long Le, Swaroop Mishra, Hossein Mobahi, Jindong Gu, Zifeng Wang, Hootan Nakhost, Chitta Baral, Chen-Yu Lee, Tomas Pfister, Hamid Palangi
| Challenge: | Existing methods for natural planning lack constraint-guided iterative verification and adaptive selection . a recent study found that LLMs are not good at such planning. |
| Approach: | They propose a model-agnostic and easily scalable agent framework with three key components: constraint, verification, and selection agents. |
| Outcome: | The proposed framework improves inference-time algorithms on NATURAL PLAN and OlympiadBench benchmarks. |
InstructExcel: A Benchmark for Natural Language Instruction in Excel (2023.findings-emnlp)
Copied to clipboard
Justin Payan, Swaroop Mishra, Mukul Singh, Carina Negreanu, Christian Poelitz, Chitta Baral, Subhro Roy, Rasika Chakravarthy, Benjamin Van Durme, Elnaz Nouri
| Challenge: | Large Language Models (LLMs) can solve increasingly complex NLP tasks such as Excel specific tasks. |
| Approach: | They propose a large-scale benchmark to test whether Large Language Models can generate code that solves Excel specific tasks provided via natural language user instructions. |
| Outcome: | The proposed model outperforms existing models and provides a hard benchmark for state of the art models like GPT-4. |
Self-Instruct: Aligning Language Models with Self-Generated Instructions (2023.acl-long)
Copied to clipboard
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, Hannaneh Hajishirzi
| Challenge: | Large “instruction-tuned” language models depend heavily on human-written instruction data . this limited quantity, diversity, and creativity hinders the generality of the tuned model . |
| Approach: | They propose a framework for improving instruction-following capabilities of pretrained language models by bootstrapping off their own generations. |
| Outcome: | The proposed framework outperforms existing public instruction datasets by 5% . it generates instructions, input, and output samples, then filters invalid or similar ones . |
Real-Time Visual Feedback to Guide Benchmark Creation: A Human-and-Metric-in-the-Loop Workflow (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent research has shown that language models exploit ‘artifacts’ in benchmarks to solve tasks, rather than learning them, leading to inflated model performance. |
| Approach: | They propose a benchmark creation paradigm for NLP that focuses on guiding crowdworkers and provides realtime visual feedback to improve sample quality. |
| Outcome: | The proposed paradigm decreases effort, frustration, mental, and temporal demands of crowdworkers and analysts, while increasing the performance of both user groups. |
Towards Robust Mathematical Reasoning (2025.emnlp-main)
Copied to clipboard
Thang Luong, Dawsen Hwang, Hoang H Nguyen, Golnaz Ghiasi, Yuri Chervonyi, Insuk Seo, Junsu Kim, Garrett Bingham, Jonathan Lee, Swaroop Mishra, Alex Zhai, Huiyi Hu, Henryk Michalewski, Jimin Kim, Jeonghyun Ahn, Junhwi Bae, Xingyou Song, Trieu Hoang Trinh, Quoc V Le, Junehyuk Jung
| Challenge: | IMO-Bench is a suite of advanced reasoning benchmarks that targets the international mathematical Olympiad level. |
| Approach: | They propose IMO-Bench, a suite of advanced reasoning benchmarks that targets the level of the international mathematical Olympiad. |
| Outcome: | IMO-Bench is a suite of advanced reasoning benchmarks that targets the level of the international mathematical Olympiad. |
Cross-Task Generalization via Natural Language Crowdsourcing Instructions (2022.acl-long)
Copied to clipboard
| Challenge: | Despite the success of supervised learning, models often struggle with generalization across tasks. |
| Approach: | They propose to use crowdsourcing instructions to build a model that learns a new task by understanding the human-readable instructions that define it. |
| Outcome: | The proposed model can learn from seen tasks and generalize to unseen tasks given its natural crowdsourcing instructions. |
Generalized but not Robust? Comparing the Effects of Data Modification Methods on Out-of-Domain Generalization and Adversarial Robustness (2022.findings-acl)
Copied to clipboard
| Challenge: | Data modification has been proposed as an effective solution for generalizing to out-of-domain (OOD) inputs. |
| Approach: | They propose to use data modification to generalize to out-of-domain inputs . they also analyze their adversarial robustness using a synthetic dataset . |
| Outcome: | The proposed data modification strategies improve OOD accuracy and AR, but data filtering hurts OOD on other tasks. |
LILA: A Unified Benchmark for Mathematical Reasoning (2022.emnlp-main)
Copied to clipboard
Swaroop Mishra, Matthew Finlayson, Pan Lu, Leonard Tang, Sean Welleck, Chitta Baral, Tanmay Rajpurohit, Oyvind Tafjord, Ashish Sabharwal, Peter Clark, Ashwin Kalyan
| Challenge: | Towards evaluating and improving AI systems in this domain, we propose a mathematical reasoning benchmark based on 23 diversetasks . |
| Approach: | They propose a mathematical reasoning benchmark that includes 23 diverse tasks . they extend the benchmark by collecting task instructions and solutions in the form of Python programs . |
| Outcome: | The proposed model improves on multi-tasking while the best performing model only achieves 60.40%. |
Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks (2022.emnlp-main)
Copied to clipboard
Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Lai, Ishan Purohit, Ishani Mondal, Jacob Anderson, Kirby Kuznia, Krima Doshi, Kuntal Kumar Pal, Maitreya Patel, Mehrad Moradshahi, Mihir Parmar, Mirali Purohit, Neeraj Varshney, Phani Rohitha Kaza, Pulkit Verma, Ravsehaj Singh Puri, Rushang Karia, Savan Doshi, Shailaja Keyur Sampat, Siddhartha Mishra, Sujan Reddy A, Sumanta Patro, Tanay Dixit, Xudong Shen
| Challenge: | a benchmark of 1,616 diverse NLP tasks and their expert-written instructions is used to test generalization of models to unseen tasks . a recent study shows that instruction-following models outperform instruction-based models by over 9% . |
| Approach: | They build a benchmark of 1,616 diverse NLP tasks and their expert-written instructions. |
| Outcome: | The proposed model outperforms existing instruction-following models by over 9% on the benchmark despite being smaller. |
Constructing Flow Graphs from Procedural Cybersecurity Texts (2021.findings-acl)
Copied to clipboard
| Challenge: | Graph Convolution Network with BERT sentence embeddings outperforms BERT in all three domains. |
| Approach: | They propose to identify relevant information from procedural texts and generate information flows between sentences from them. |
| Outcome: | The proposed method outperforms BERT in two domains of cybersecurity. |
Reframing Instructional Prompts to GPTk’s Language (2022.findings-acl)
Copied to clipboard
| Challenge: | Using reframing techniques, we find that instructional prompts are easier to follow for Language Models (LMs) |
| Approach: | They propose reframing techniques for manual reformulation of prompts into more effective ones . they compare performance of LMs prompted with reframed instructions on 12 NLP tasks . |
| Outcome: | The reframing techniques used for prompt reformulation improve performance on 12 tasks . the techniques boost performance on LMs with different sizes compared with original prompts . |
“John is 50 years old, can his son be 65?” Evaluating NLP Models’ Understanding of Feasibility (2023.eacl-main)
Copied to clipboard
Himanshu Gupta, Neeraj Varshney, Swaroop Mishra, Kuntal Kumar Pal, Saurabh Arjun Sawant, Kevin Scaria, Siddharth Goyal, Chitta Baral
| Challenge: | Recent work has found that large-scale language models lack commonsense reasoning ability . a dataset evaluating large-level language models is needed to evaluate their understanding of feasibility . |
| Approach: | They propose a question-answering dataset that tests understanding of feasibility . they propose to use commonsense reasoning to reason about when an action is feasible . |
| Outcome: | The proposed dataset shows that state-of-the-art models struggle to answer feasibility questions correctly. |
NumGLUE: A Suite of Fundamental yet Challenging Mathematical Reasoning Tasks (2022.acl-long)
Copied to clipboard
Swaroop Mishra, Arindam Mitra, Neeraj Varshney, Bhavdeep Sachdeva, Peter Clark, Chitta Baral, Ashwin Kalyan
| Challenge: | Existing AI systems fail to perform basic mathematical reasoning when presented in a slightly different manner. |
| Approach: | They propose a multi-task benchmark that evaluates the performance of AI systems on eight different tasks that at their core require simple arithmetic understanding. |
| Outcome: | The proposed benchmark compares the performance of AI systems on eight different tasks. |
InstructABSA: Instruction Learning for Aspect Based Sentiment Analysis (2024.naacl-short)
Copied to clipboard
| Challenge: | Experimental results on the Sem Eval 2014, 15, and 16 datasets demonstrate that InstructABSA outperforms the previous state-of-the-art (SOTA) approaches on Term Extraction (ATE), Sentiment Classification(ATSC) and Sentimence Pair Extraction(ASPE) subtasks. |
| Approach: | They introduce positive, negative, and neutral examples to each training sample, and instruction tune the model (Tk-Instruct) for ABSA subtasks. |
| Outcome: | The proposed model outperforms the state-of-the-art (SOTA) on Term Extraction (ATE), Sentiment Classification (ATSC) and Sentimence Pair Extractions (ASPE) subtasks. |
HELP ME THINK: A Simple Prompting Strategy for Non-experts to Create Customized Content with Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing prompting techniques for providing control are task-specific and lack generality; this limits their adoption among non-expert users. |
| Approach: | They propose a prompting strategy that encourages large language models to help non-expert users by asking relevant questions and leveraging user answers to execute a task. |
| Outcome: | The proposed prompting strategy is able to help non-expert users with a variety of tasks. |
Investigating Selective Prediction Approaches Across Several Tasks in IID, OOD, and Adversarial Settings (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to selective prediction are not consistently outperform the simplest baseline MaxProb in all three settings. |
| Approach: | They propose to use a large-scale setup of 17 datasets to study selective prediction in NLP tasks using in-domain, out-of-domain and adversarial settings. |
| Outcome: | The proposed approaches outperform the simplest baseline MaxProb in in-domain, out-of-domain and adversarial settings, but none consistently outperformed in all three settings. |