Papers by Ashutosh Joshi
Towards Robust Evaluation of Unlearning in LLMs via Data Transformations (2024.findings-emnlp)
Copied to clipboard
Abhinav Joshi, Shaswati Saha, Divyaksh Shukla, Sriram Vema, Harsh Jhamtani, Manas Gaur, Ashutosh Modi
| Challenge: | Large Language Models (LLMs) have shown to be a great success in a wide range of applications ranging from regular NLP-based use cases to AI agents. |
| Approach: | They examine the robustness of existing MUL techniques for their ability to enable leakage-proof forgetting in LLMs. |
| Outcome: | The proposed methods can be used to enable leakage-proof forgetting in LLMs. |
From Representation to Choice: Tracing Decision Emergence Across Languages in LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in large language models have made them highly multilingual, but how they internally reason remains unexplored. |
| Approach: | They propose to model multilingual reasoning through a decision-making perspective using aligned multiple-choice questions from the mMMLU benchmark. |
| Outcome: | The proposed model shows that languages share similar activation spaces, but subtle divergences emerge as decisions propagate through transformer layers. |
Calibration Across Layers: Understanding Calibration Evolution in LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated inherent calibration capabilities, where predicted probabilities align well with correctness . previous studies have linked this behavior to specific components in the final layer, such as entropy neurons and the unembedding matrix’s null space. |
| Approach: | They propose to examine how calibration evolves throughout the network's depth. |
| Outcome: | The proposed calibration direction improves calibration metrics without harming accuracy. |
Towards Quantifying Commonsense Reasoning with Mechanistic Insights (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent studies have evaluated commonsense reasoning abilities using text-based tasks. |
| Approach: | They propose to capture commonsense knowledge in a graphical representation of 37 daily human activities in graphical form and frame them to frame commonsensical queries. |
| Outcome: | The proposed model can frame an enormous number of commonsense queries ( 10 17) and perform rigorous evaluations of common sense reasoning in LLMs. |
IL-TUR: Benchmark for Indian Legal Text Understanding and Reasoning (2024.acl-long)
Copied to clipboard
| Challenge: | Legal systems worldwide struggle with exponentially growing legal cases in various courts. |
| Approach: | They propose a benchmark for Indian legal text understanding and reasoning task that includes domain-specific tasks that address different aspects of the legal system. |
| Outcome: | The proposed benchmark for Indian legal text understanding and reasoning aims to address the gap between models and the ground truth. |
CISLR: Corpus for Indian Sign Language Recognition (2022.emnlp-main)
Copied to clipboard
Abhinav Joshi, Ashwani Bhat, Pradeep S, Priya Gole, Shashwat Gupta, Shreyansh Agarwal, Ashutosh Modi
| Challenge: | Existing work on natural language processing has shown promising improvements in text classification, translation and generation in widely used spoken languages. |
| Approach: | They propose a new Indian Sign Language corpus for word-level recognition using videos . they propose CISLR model that leverages resource rich American Sign Language to learn generalized features for improving Indian Sign language predictions. |
| Outcome: | The proposed model improves word recognition in Indian Sign Language using video . it leverages resource rich American Sign Language to learn generalized features . |
Augmenting Training Data for Massive Semantic Matching Models in Low-Traffic E-commerce Stores (2022.naacl-industry)
Copied to clipboard
Ashutosh Joshi, Shankar Vishwanath, Choon Teo, Vaclav Petricek, Vishy Vishwanathan, Rahul Bhagat, Jonathan May
| Challenge: | Existing methods to augment training data for e-commerce stores using behavioral data are limited in low-traffic stores . eXtreme multi-label classification systems require large amounts of customer behavior data . |
| Approach: | They propose a technique that augments behavioral training data via query reformulation . they use an example semantic matching model from the e-commerce store AL-XMC . |
| Outcome: | The proposed method improves quality of the AL-XMC model over a baseline model. |
Linking Knowledge to Care: Knowledge Graph-Augmented Medical Follow-Up Question Generation (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing large language models (LLMs) fail to identify information gaps across diverse symptoms. |
| Approach: | They propose a Knowledge Graph-augmented LLM with active in-context learning to generate relevant and important follow-up questions. |
| Outcome: | The proposed framework outperforms state-of-the-art methods by 5% - 8% on relevant benchmarks. |
CheckersGPT: Learning World Models through Language Modeling (2024.acl-srw)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown impressive performance on various tasks, but the underlying process behind predicting the desired next token remains a black box. |
| Approach: | They train a GPT-style autoregressive language model using only the next character prediction objective and then train corresponding model with different layer sizes. |
| Outcome: | The proposed model shows a hint of learning a world model representation of the board positions on a simulated game of checkers and human gameplay dataset. |
ISLTranslate: Dataset for Translating Indian Sign Language (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing sign language translation datasets are limited for the Indian sign language. |
| Approach: | They propose to use ISLTranslate to create a sign language translation dataset for Indian Sign Language consisting of 31k ISL-English sentence/phrase pairs. |
| Outcome: | The proposed dataset is the largest for Indian Sign Language translation dataset . it compares with a transformer-based model to validate the performance of existing systems. |
PoseStitch-SLT: Linguistically Inspired Pose-Stitching for End-to-End Sign Language Translation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Sign language translation remains a challenging task due to the scarcity of large-scale, sentence-aligned datasets. |
| Approach: | They propose a pose-based pre-training scheme that is inspired by a linguistic-templates-based sentence generation technique. |
| Outcome: | The proposed pre-training scheme outperforms state-of-the-art methods for pose-based gloss-free translation on two sign language datasets. |
U-CREAT: Unsupervised Case Retrieval using Events extrAcTion (2023.acl-long)
Copied to clipboard
| Challenge: | Prior Case Retrieval (PCR) is about automatically citing relevant prior legal cases in a given query case. |
| Approach: | They propose a new benchmark for prior case retrieval based on a legal query case . they propose an unsupervised retrieval method-based pipeline U-CREAT . |
| Outcome: | The proposed method significantly improves performance and makes retrieval faster compared to BM25. |
COGMEN: COntextualized GNN based Multimodal Emotion recognitioN (2022.naacl-main)
Copied to clipboard
| Challenge: | During a conversation, a person’s emotions are influenced by the other speaker’s utterances and their own emotional state over the utterrances. |
| Approach: | They propose a Graph Neural Network based Multi-modal Emotion recognitioN system that leverages local and global information in a conversation. |
| Outcome: | The proposed system gives state-of-the-art results on IEMOCAP and MOSEI datasets and detailed ablation experiments show the importance of modeling information at both levels. |
Deep Metric Learning to Hierarchically Rank - An Application in Product Retrieval (2023.emnlp-industry)
Copied to clipboard
Kee Kiat Koo, Ashutosh Joshi, Nishaanth Reddy, Karim Bouyarmane, Ismail Tutar, Vaclav Petricek, Changhe Yuan
| Challenge: | e-commerce search engines use customer behavior signals to augment lexical matching and improve search relevance. |
| Approach: | They propose a method to identify duplicate and near-duplicate products across stores . they use Hierarchical Ranked Multi Similarity Loss to learn hierarchical metric space . |
| Outcome: | The proposed model outperforms baselines in terms of catalog coverage and precision of the mappings. |
iSign: A Benchmark for Indian Sign Language Processing (2024.findings-acl)
Copied to clipboard
Abhinav Joshi, Romit Mohanty, Mounika Kanakanti, Andesha Mangla, Sudeep Choudhary, Monali Barbate, Ashutosh Modi
| Challenge: | Indian Sign Language has limited resources for developing machine learning and data-driven approaches for automated language processing. |
| Approach: | They propose to use a sign language dataset to provide a benchmark for Indian Sign Language processing. |
| Outcome: | The proposed benchmarks will help improve sign language translation models and open up various ways for advancing natural language processing. |
Striking a Balance: Alleviating Inconsistency in Pre-trained Models for Symmetric Classification Tasks (2022.findings-acl)
Copied to clipboard
| Challenge: | Inconsistency is observed in symmetric classification tasks that take two inputs and require the output to be invariant of the order of the inputs. |
| Approach: | They propose a consistency loss function to alleviate inconsistency in symmetric classification tasks that take two inputs and require the output to be invariant of the order of the inputs. |
| Outcome: | The proposed model improves consistency in predictions for three paraphrase detection datasets without significant drop in accuracy scores. |