Papers by Nitish Gupta
Event Linking: Grounding Event Mentions to Wikipedia (2023.eacl-main)
Copied to clipboard
| Challenge: | a new task for natural language understanding is called Event Linking . the context where an event is mentioned lacks the details of this event . |
| Approach: | They propose a new task to link an article's event mention to the most appropriate Wikipedia page . they collect a training set from Wikipedia and evaluate two models to test the task . |
| Outcome: | The proposed model is based on a dataset and a real-world news domain . it is expected that the most appropriate Wikipedia page will provide rich knowledge about the mention . |
Paired Examples as Indirect Supervision in Latent Decision Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | a new method to learn compositional structured models is needed . end-task supervision provides only a weak indirect signal on values the latent decisions should take. |
| Approach: | They propose a way to leverage paired examples that provide stronger cues for learning latent decisions . they use a DROP dataset to acquire paired questions that provide strong cue signals . |
| Outcome: | The proposed approach improves compositional question answering on a DROP dataset. |
IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages (2024.acl-long)
Copied to clipboard
| Challenge: | IndicGenBench is the largest benchmark for evaluating large language models on user-facing generation tasks across a diverse set of 29 Indic languages . |
| Approach: | They evaluate large language models on user-facing generation tasks across 29 languages . they use human curation to provide multi-way parallel evaluation data for many under-represented languages a github repository . |
| Outcome: | IndicGenBench is the largest benchmark for evaluating LLMs on user-facing generation tasks across a diverse set of 29 Indic languages covering 13 scripts and 4 language families. |
Enforcing Consistency in Weakly Supervised Semantic Parsing (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for training semantic parsers from only (utterance, denotation) supervision are challenging. |
| Approach: | They propose to use consistency between output programs for related inputs to reduce the impact of spurious programs. |
| Outcome: | The proposed formalisms improve model performance even without consistency-based training. |
What do we expect from Multiple-choice QA Systems? (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work has shown that good performance on a dataset might not correlate well with human’s expectations from models that “understand” language. |
| Approach: | They propose to train a top performing multiple choice question answering model against expectations from models that "understand" language. |
| Outcome: | The proposed training paradigm leads to a model that performs on par with the original model while better satisfying our expectations. |
Joint Multilingual Supervision for Cross-lingual Entity Linking (D18-1)
Copied to clipboard
| Challenge: | Entity Linking (XEL) systems ground entity mentions written in any language to Wikipedia . XEL is challenging for most languages due to limited availability of resources as supervision . |
| Approach: | They develop a cross-lingual XEL approach that combines supervision from multiple languages jointly. |
| Outcome: | The proposed approach significantly improves on the current state-of-the-art in 8 languages. |
XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languages (2023.findings-emnlp)
Copied to clipboard
Sebastian Ruder, Jonathan Clark, Alexander Gutkin, Mihir Kale, Min Ma, Massimo Nicosia, Shruti Rijhwani, Parker Riley, Jean-Michel Sarr, Xinyi Wang, John Wieting, Nitish Gupta, Anna Katanova, Christo Kirov, Dana Dickinson, Brian Roark, Bidisha Samanta, Connie Tao, David Adelani, Vera Axelrod, Isaac Caswell, Colin Cherry, Dan Garrette, Reeve Ingle, Melvin Johnson, Dmitry Panteleev, Partha Talukdar
| Challenge: | Existing datasets are often informed by established research directions in the NLP community. |
| Approach: | They propose a benchmark to evaluate the capabilities of language models across 88 under-represented languages over 9 key user-centric technologies including ASR, OCR, MT, and information access tasks. |
| Outcome: | The proposed benchmark evaluates the capabilities of language models across 88 under-represented languages over 9 key user-centric technologies including ASR, OCR, MT, and information access tasks. |
Improving Compositional Generalization in Semantic Parsing (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Generalization of models to out-of-distribution data has sparked substantial interest . compositional generalization is the ability to systematically generalize to test examples composed of components seen during training . |
| Approach: | They propose to extend compositional generalization in semantic parsing by using contextual representations and training attention to agree with pre-computed token alignments. |
| Outcome: | The proposed extensions improve compositional generalization on OOD compositions. |
Neural Compositional Denotational Semantics for Question Answering (D18-1)
Copied to clipboard
| Challenge: | a new model for compositional questions is needed to answer multi-step reasoning . the model is inspired by formal approaches to compositional semantics . |
| Approach: | They propose an end-to-end differentiable model for interpreting compositional questions . they build a latent tree of interpretable expressions over a sentence . |
| Outcome: | The proposed model outperforms RNN encoders when test questions are longer than training questions. |
Evaluating Models’ Local Decision Boundaries via Contrast Sets (2020.findings-emnlp)
Copied to clipboard
Matt Gardner, Yoav Artzi, Victoria Basmov, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hannaneh Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noah A. Smith, Sanjay Subramanian, Reut Tsarfaty, Eric Wallace, Ally Zhang, Ben Zhou
| Challenge: | Standard test sets for supervised learning evaluate in-distribution generalization but are misleading when a dataset has systematic gaps. |
| Approach: | They propose a more rigorous annotation paradigm for NLP that helps to close systematic gaps in the test data. |
| Outcome: | The proposed model performs significantly lower on contrast sets than on the original test sets—up to 25% in some cases. |
Obtaining Faithful Interpretations from Compositional Neural Networks (2020.acl-main)
Copied to clipboard
Sanjay Subramanian, Ben Bogin, Nitish Gupta, Tomer Wolfson, Sameer Singh, Jonathan Berant, Matt Gardner
| Challenge: | Neural module networks (NMNs) are a popular approach for modeling compositionality but prior work implicitly assumed that the structure of the network modules provides a faithful explanation of the model’s reasoning. |
| Approach: | They propose to use auxiliary supervision to train a model with a structured model that can understand the reasoning process and make better choices for module architecture. |
| Outcome: | The proposed models on two datasets show that the proposed models do not provide a faithful explanation of model behaviour. |
Bootstrapping Multilingual Semantic Parsers using Large Language Models (2023.eacl-main)
Copied to clipboard
| Challenge: | Despite cross-lingual generalization, translation models require significant amounts of labeled data for many low-resource languages . brittle translation services may be due to domain mismatch between input text and general-purpose text . |
| Approach: | They propose to use large language models to translate English datasets into several languages via few-shot prompting. |
| Outcome: | The proposed method outperforms a strong translation-train baseline on 41 out of 50 languages. |
Overestimation of Syntactic Representation in Neural Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Several testing methodologies have been developed to probe models’ syntactic representations. |
| Approach: | They propose a method to determine syntactic structure by training a model on strings generated according to a template and testing its ability to distinguish between similar ones with different syntax. |
| Outcome: | The proposed method reproduces positive results with two non-syntactic baseline language models: an n-gram model and an LSTM model trained on scrambled inputs. |