Papers by Sandipan Dandapat
Multi Task Learning For Zero Shot Performance Prediction of Multilingual Models (2022.acl-long)
Copied to clipboard
| Challenge: | Massively Multilingual Transformer based Language Models have been shown to be effective on zero-shot transfer across languages, though performance varies from language to language depending on pivot language(s) used for fine-tuning. |
| Approach: | They propose to combine multi-task learning problems with multi-lingual Transformers to model zero-shot transfer across languages. |
| Outcome: | The proposed model can predict zero-shot transfer across languages with a multi-task learning problem with pretraining data in very few languages. |
”Diversity and Uncertainty in Moderation” are the Key to Data Selection for Multilingual Few-shot Transfer (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches for few-shot transfer show significant gain over zero-shot transfers . language resource distribution is skewed across the world's languages . proposed methods use multiple measures such as data entropy and gradient embedding . |
| Approach: | They propose a loss embedding method for sequence labeling tasks that induces diversity and uncertainty sampling similar to gradient embeddment. |
| Outcome: | The proposed methods outperform baseline methods for POS tagging, NER, and NLI tasks for up to 20 languages. |
DiTTO: A Feature Representation Imitation Approach for Improving Cross-Lingual Transfer (2023.eacl-main)
Copied to clipboard
| Challenge: | Zero-shot cross-lingual transfer has been shown to be sub-optimal across low-resource languages due to the skew in resource distribution in languages. |
| Approach: | They propose to jointly reduce feature incongruity between the source and target language and increase generalization capabilities of pre-trained multilingual transformers. |
| Outcome: | Empirical results show that the proposed approach outperforms the standard zero-shot fine-tuning method on multiple datasets across all languages using only unlabeled instances in the target language. |
Language Modeling for Code-Mixing: The Role of Linguistic Theory based Synthetic Data (P18-1)
Copied to clipboard
| Challenge: | Code-mixed (CM) language training is a difficult problem because of lack of data and the increased confusability due to the presence of more than one language. |
| Approach: | They propose a computational technique for creating grammatically valid artificial CM data based on the Equivalence Constraint Theory. |
| Outcome: | The proposed method reduces the perplexity of the model and does not reduce the perceptibility of the models. |
On the Economics of Multilingual Few-shot Learning: Modeling the Cost-Performance Trade-offs of Machine Translated and Manual Data (2022.naacl-main)
Copied to clipboard
| Challenge: | a framework to evaluate the performance and cost trade-offs between machine-translated and manually-created labelled data is presented. |
| Approach: | They propose a framework to evaluate the performance and cost trade-offs between machine-translated and manually-created labelled data for task-specific fine-tuning of massively multilingual language models. |
| Outcome: | The proposed framework can be used to evaluate cost trade-offs between machine-translated and manually-created labelled data for task-specific fine-tuning of massively multilingual models. |
On the Calibration of Massively Multilingual Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Massively Multilingual Language Models (MMLMs) have gained popularity due to their effectiveness in cross-lingual transfer. |
| Approach: | They investigate how well calibrated MMLMs are with respect to confidence . they find that calibration methods like temperature scaling and label smoothing improve calibration . |
| Outcome: | The proposed models are able to generalize in languages unseen during fine-tuning, but they are not reliable across languages. |
Vector Space Interpolation for Query Expansion (2022.aacl-short)
Copied to clipboard
| Challenge: | Topic-sensitive query set expansion is crucial for queries related to sensitive and emerging topics. |
| Approach: | They propose a method for topic-sensitive query set expansion using vector space interpolation. |
| Outcome: | The proposed method generates new queries about the sensitive topic by incorporating set diversity, which is not captured by traditional sentence-level augmentation methods such as paraphrasing or back-translation. |
Multilingual CheckList: Generation and Evaluation (2022.findings-aacl)
Copied to clipboard
Karthikeyan K, Shaily Bhatt, Pankaj Singh, Somak Aditya, Sandipan Dandapat, Sunayana Sitaram, Monojit Choudhury
| Challenge: | Multilingual evaluation benchmarks usually contain limited high-resource languages and do not test models for specific linguistic capabilities. |
| Approach: | They propose an algorithm for automatically extracting target language CheckList templates from machine translated instances of a source language templates. |
| Outcome: | The proposed algorithm compares with CheckLists created with human verification in Hindi and 9 other languages. |
SAGE: A Generic Framework for LLM Safety Evaluation (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Current safety evaluation methodologies focus on single-turn interactions with generic policies, failing to capture conversational dynamics of real-world usage and application-specific harms. |
| Approach: | They propose a framework for customized and dynamic harm evaluations that employs prompted adversarial agents with diverse personalities based on the Big Five model. |
| Outcome: | The proposed framework enables system-aware multi-turn conversations that adapt to target applications and harm policies. |
GLUECoS: An Evaluation Benchmark for Code-Switched NLP (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies show multilingual contextual embedding models perform better on cross-lingual and multilingual tasks. |
| Approach: | They propose to evaluate multilingual contextual embedding models on multilingual data . they use language identification from text, POS tagging, Named Entity Recognition and Question Answering . |
| Outcome: | The proposed benchmark evaluates models on language identification from text, POS tagging, Named Entity Recognition, Question Answering and a new task for code-switching, Natural Language Inference. |
INMT: Interactive Neural Machine Translation Prediction (D19-3)
Copied to clipboard
| Challenge: | Existing MT systems are only useful for information assimilation, and require substantial manual post processing. |
| Approach: | They propose an Interactive Machine Translation interface that assists human translators with on-the-fly hints and suggestions. |
| Outcome: | The proposed interface makes the end-to-end translation process faster, more efficient and creates high-quality translations. |
INMT-Lite: Accelerating Low-Resource Language Data Collection via Offline Interactive Neural Machine Translation (2024.lrec-main)
Copied to clipboard
Harshita Diddee, Anurag Shukla, Tanuja Ganu, Vivek Seshadri, Sandipan Dandapat, Monojit Choudhury, Kalika Bali
| Challenge: | Interactive Neural Machine Translation (INMT) systems can be used to promote data collection in several under-resourced languages, but are often not adapted to the deployment constraints native language speakers operate in. |
| Approach: | They propose to use interactive neural machine translation systems to promote data collection in several under-resourced languages by integrating three different modes of Internet-independent deployment and four assistive interfaces suitable for data-sparse languages. |
| Outcome: | The proposed model improves the data generation experience of community members along multiple axes without compromising on the quality of the generated translations. |
Do Language Models Have a Common Sense regarding Time? Revisiting Temporal Commonsense Reasoning in the Era of Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Temporal reasoning is a vital component of human communication and understanding, yet remains an underexplored area within the context of Large Language Models (LLMs). |
| Approach: | They propose to use 3 prompting strategies to evaluate 8 different LLMs across 6 datasets and 2 Code Generation LMs to perform the analysis. |
| Outcome: | The proposed models perform better on NLP tasks than the standard models on the same dataset. |
Processing and Understanding Mixed Language Data (D19-2)
Copied to clipboard
| Challenge: | Multilingual communities exhibit code-mixing, mixing of two or more languages in a single conversation . social media and other informal interactive platforms are allowing code-switching in user-generated text . |
| Approach: | a tutorial aims to provide a foundation for researchers to study code-mixing in multilingual communities. |
| Outcome: | a tutorial aims to provide new researchers with a foundation in linguistics and computational aspects of code-mixing. |
Identifying Transferable Information Across Domains for Cross-domain Sentiment Classification (P18-1)
Copied to clipboard
| Challenge: | Cross-domain sentiment classification is challenging due to polarity orientation and significance differences . supervised learning algorithms have to be re-trained on every new domain . |
| Approach: | They propose that words that do not change their polarity and significance represent transferable information across domains for cross-domain sentiment classification. |
| Outcome: | The proposed method improves cross-domain sentiment classification performance by identifying polarity-preserving significant words across domains. |
Uncovering Stereotypes in Large Language Models: A Task Complexity-based Approach (2024.eacl-long)
Copied to clipboard
| Challenge: | Recent Large Language Models (LLMs) have unlocked unprecedented applications of AI. |
| Approach: | They propose to use a social benchmark to evaluate the bias protection provided by Large Language Models (LLMs) with a variety of tasks with varying complexities to assess their effectiveness. |
| Outcome: | The proposed benchmark shows that both ChatGPT and GPT-4 have strong biases with respect to nationality, gender, race, and religion. |
LLM Safety for Children (2025.naacl-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly impacting children through education, toys, and therapy, offering benefits like improved mental health and parental controls. |
| Approach: | They propose a comprehensive approach to evaluating LLM safety specifically for children by listing potential risks that children may encounter when using LLM-powered applications. |
| Outcome: | The proposed model bridges the gap in child safety literature across various fields. |
Translating Web Search Queries into Natural Language Questions (L18-1)
Copied to clipboard
| Challenge: | a new method to generate natural language questions from keyword-based queries is proposed . a synergy between query-to-question problem and standard machine translation (MT) model is found . |
| Approach: | They propose a method to generate well-formed natural language questions from keyword-based queries. |
| Outcome: | The proposed method is well-formed natural language question generated from keyword-based query. |
Performance and Risk Trade-offs for Multi-word Text Prediction at Scale (2023.findings-eacl)
Copied to clipboard
Aniket Vashishtha, S Sai Prasad, Payal Bajaj, Vishrav Chaudhary, Kate Cook, Sandipan Dandapat, Sunayana Sitaram, Monojit Choudhury
| Challenge: | Large Language Models (LLMs) generate ethically inappropriate texts even for seemingly innocuous contexts. |
| Approach: | They propose to use large language models to detect and filter toxic content in text prediction tasks by evaluating their toxicity detection approaches against a manually crafted CheckList of harms. |
| Outcome: | The proposed methods are compared against a checklist of harms targeted at different groups and different levels of severity in English. |
Enhancing Zero-shot Chain of Thought Prompting via Uncertainty-Guided Strategy Selection (2025.coling-main)
Copied to clipboard
Shanu Kumar, Saish Mendke, Karody Lubna Abdul Rahman, Santosh Kurasa, Parag Agrawal, Sandipan Dandapat
| Challenge: | Existing methods for chain-of-thought (CoT) prompting are limited by handcrafted demonstrations and trigger phrases are prone to inaccuracies. |
| Approach: | They propose a method that generates rationales using a trigger phrase to select effective demonstrations without accessing model parameters. |
| Outcome: | The proposed method outperforms existing methods across four reasoning benchmarks and is robust and scalable. |