Papers by Sunita Sarawagi
Topic Sensitive Attention on Generic Corpora Corrects Sense Bias in Pretrained Embeddings (P19-1)
Copied to clipboard
| Challenge: | Existing methods to adapt pretrained embeddings to a large corpus are limited and do not provide sufficient quality. |
| Approach: | They propose to use a small corpus D_T to pretrain embeddings that accurately capture the sense of words in a limited set of focused topics. |
| Outcome: | The proposed embeddings capture the sense of words in a topic in spite of the limited size of the corpus D_T. |
RECAST: Retrieval-Augmented Contextual ASR via Decoder-State Keyword Spotting (2025.findings-emnlp)
Copied to clipboard
| Challenge: | RECAST is a lightweight retrieval-augmented approach for contextual ASR . it repurposes decoder states of a pretrained ASR model to retrieve relevant keywords . |
| Approach: | RECAST is a retrieval-augmented approach that repurposes decoder states of a pretrained ASR model to retrieve relevant keywords without requiring audio exemplars. |
| Outcome: | RECAST outperforms full-list prompt biasing and strong phonetic/text baselines on 4,000 keywords across diverse domains. |
Overlap-based Vocabulary Generation Improves Cross-lingual Transfer Among Related Languages (2022.acl-long)
Copied to clipboard
| Challenge: | Pre-trained multilingual models have shown great potential for zero-shot cross-lingual transfer to low web-resource languages (LRLs). |
| Approach: | They propose a vocabulary generation algorithm which enhances lexical overlap across related languages by generating a token that increases the representation of LRLs. |
| Outcome: | The proposed approach improves cross-lingual transfer accuracy without reducing HRL representation and accuracy. |
Exploiting Language Relatedness for Low Web-Resource Language Model Adaptation: An Indic Languages Study (2021.acl-long)
Copied to clipboard
Yash Khemchandani, Sarvesh Mehtani, Vaidehi Patil, Abhijeet Awasthi, Partha Talukdar, Sunita Sarawagi
| Challenge: | Recent research in multilingual language models (LMs) has demonstrated their ability to effectively handle multiple languages in a single model. |
| Approach: | They propose to exploit relatedness among languages in a language family to overcome corpora limitations of LRLs. |
| Outcome: | The proposed model exploits relatedness among languages in a language family to overcome corpora limitations for low web-resource languages. |
Training Data Augmentation for Code-Mixed Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | We show a 5.8 point increase in BLEU on heavily code-mixed sentences . code-mixing is becoming more commonplace in several bilingual communities . |
| Approach: | They propose a method to convert existing parallel data sources into code-mixed parallel data. |
| Outcome: | The proposed method shows a 5.8 point increase in BLEU on heavily code-mixed sentences on a Hindi-English code-mixed translation task. |
Diverse In-Context Example Selection After Decomposing Programs and Aligned Utterances Improves Semantic Parsing (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are well suited for seq2seq translation . a lack of pretraining corpora can hinder the use of LLMs for structured interpretation . |
| Approach: | They propose to decompose available ICE trees into fragments and use additional invocations to map them to corresponding utterances. |
| Outcome: | The proposed method shows visible gains on diverse parsing benchmarks on popular languages. |
Efficient Training of Language Models with Compact and Consistent Next Token Distributions (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to train language models have focused on maximizing the likelihood of the next token . however, the construction and querying of such n-grams can be costly and impede training speed. |
| Approach: | They propose a method to train language models faster by pre-aggregating corpus with collapsed n-gram distribution. |
| Outcome: | The proposed model improves model quality and convergence rate while reducing variance across mini-batches compared to the standard next-token loss method. |
Surprisingly Easy Hard-Attention for Sequence to Sequence Learning (D18-1)
Copied to clipboard
| Challenge: | Existing attention mechanisms are hard and hard, but they are more accurate when trained. |
| Approach: | They propose to use a beam approximation of the joint distribution between attention and output to train sequence to sequence learning. |
| Outcome: | The proposed method is compared to existing attention mechanisms on five translation tasks and shows consistent gains on the same tasks. |
CRUSH4SQL: Collective Retrieval Using Schema Hallucination For Text2SQL (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing Text-to-SQL generators require the entire schema to be encoded with the user text. |
| Approach: | They propose a method that uses an LLM to hallucinate a minimal DB schema . they use the hallucinated schema to retrieve a subset of the actual schema based on multiple dense retrievals . |
| Outcome: | The proposed method leads to significantly higher recall than existing methods. |
Parallel Iterative Edit Models for Local Sequence Transduction (D19-1)
Copied to clipboard
| Challenge: | Recent approaches to local sequence transduction are based on the popular encoder-decoder model for sequence to sequence learning. |
| Approach: | They propose a parallel iterative edit model for the problem of local sequence transduction arising in tasks like Grammatical error correction (GEC). |
| Outcome: | The proposed model is faster and more accurate than the current encoder-decoder model for local sequence transduction tasks like translation and paraphrasing. |
Quality Scoring of Source Words in Neural Translation Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent approaches to improving word-level quality scores on input source sentences require training special word-scoring models or require repeated invocation of the translation model. |
| Approach: | They propose to reason how well each word is explained by the target sentence as against the source language model and use it to translate into an unfamiliar target language. |
| Outcome: | The proposed method provides up to five points higher F1 scores and is significantly faster than the state of the art methods on three language pairs. |
Adapting Multilingual Models for Code-Mixed Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work has addressed the lack of gold standard code-mixed to pure language parallel data with data augmentation techniques. |
| Approach: | They propose a back-translation-based training scheme for code-mixed translation which eliminates dependence on external resources. |
| Outcome: | The proposed model beats previous work by up to +3.8 BLEU on code-mixed tasks. |
Robust In-Context Selection via Online Learned Position-Corrected Attention (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to fix this limitation can be classified into two ways: (1) Methods that use the LLM to generate the selection either via logits of item identifiers, or explicit rank permutations often requiring multiple LLM calls or fine-tuning. |
| Approach: | They propose a method that harnesses attention patterns available from a single forward call on the Large Language Model (LLM) the method learns the logic for item selection using a few in-context examples and a simple online position-debiasing mechanism to correct attention distortion. |
| Outcome: | The proposed method improves selection performance over direct generation and prior attention-based methods while remaining robust to prompt variations and item ordering. |
Benchmarking and Improving Text-to-SQL Generation under Ambiguity (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing decoding algorithms treat SQL queries as a string and produce unhelpful token-level diversity in the top-k. |
| Approach: | They propose a benchmarking algorithm that generates all SQLs in top-k ranked outputs . they use plan-based template generation and constrained infilling to bridge this gap . |
| Outcome: | The proposed algorithm is 2.5 times more effective than state-of-the-art models at generating all candidate SQLs in the top-k ranked outputs. |
Speech-enriched Memory for Inference-time Adaptation of ASR Models to Word Dictionaries (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing contextual biasing techniques require additional parameterization . state-of-the-art ASR systems often fail to recognize named entities or critical rare words . |
| Approach: | They propose an algorithm that uses nearest-neighbor matching to predict ASR models . a list of rare entities is indexed in a memory and then stored the best possible match . |
| Outcome: | The proposed algorithm improves the prediction of state-of-the-art ASR models on rare words . it prevents spurious matches by restricting to word-level matches . |
Accurate Online Posterior Alignments for Principled Lexically-Constrained Decoding (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods of offline alignment use only the entire target sentence. |
| Approach: | They propose a posterior alignment technique that is truly online in its execution and superior in terms of alignment error rates compared to existing methods. |
| Outcome: | The proposed technique is online in execution and superior in alignment error rates compared to existing methods. |
Diverse Parallel Data Synthesis for Cross-Database Adaptation of Text-to-SQL Parsers (2022.emnlp-main)
Copied to clipboard
| Challenge: | Adapting Text-to-SQL parsers to new database schemas is a challenging task owing to a vast diversity of schemas and zero availability of natural language queries in new schemas. |
| Approach: | They propose a framework for synthesizing parallel datasets for adapting Text-to-SQL parsers. |
| Outcome: | The proposed framework outperforms existing methods on databases with diverse schemas and zero availability of natural language queries. |
Bootstrapping Multilingual Semantic Parsers using Large Language Models (2023.eacl-main)
Copied to clipboard
| Challenge: | Despite cross-lingual generalization, translation models require significant amounts of labeled data for many low-resource languages . brittle translation services may be due to domain mismatch between input text and general-purpose text . |
| Approach: | They propose to use large language models to translate English datasets into several languages via few-shot prompting. |
| Outcome: | The proposed method outperforms a strong translation-train baseline on 41 out of 50 languages. |
NLP Service APIs and Models for Efficient Registration of New Clients (2020.findings-emnlp)
Copied to clipboard
| Challenge: | State-of-the-art NLP inference uses enormous neural architectures and models trained for GPU-months, well beyond the reach of most consumers of NLP. |
| Approach: | They propose a centralized NLP service that can be customized to suit clients . they propose NER, sentiment labeling, and predictive language modeling to improve client experience. |
| Outcome: | The proposed model can be used to improve word usage and salience across clients without re-training or fine-tuning. |