Papers by Ankit Gupta
Value-aware Approximate Attention (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approximations of dot-product attention ignore the value vectors . a value-aware objective outperforms an optimal approximate that ignores values . |
| Approach: | They propose an approximation of a value-aware objective that substantially outperforms an optimal approximate that ignores values. |
| Outcome: | The proposed value-aware objective outperforms an optimal approximation that ignores values in the context of language modeling. |
DART: Open-Domain Structured Data Record to Text Generation (2021.naacl-main)
Copied to clipboard
Linyong Nan, Dragomir Radev, Rui Zhang, Amrit Rau, Abhinand Sivaprasad, Chiachun Hsieh, Xiangru Tang, Aadit Vyas, Neha Verma, Pranav Krishna, Yangxiaokang Liu, Nadia Irwanto, Jessica Pan, Faiaz Rahman, Ahmad Zaidi, Mutethia Mutuma, Yasin Tarabar, Ankit Gupta, Tao Yu, Yi Chern Tan, Xi Victoria Lin, Caiming Xiong, Richard Socher, Nazneen Fatema Rajani
| Challenge: | Data-to-text annotations can be costly when dealing with tables with nontrivial structures. |
| Approach: | They propose a procedure for extracting semantic triples from tables that encodes their structures by exploiting table headers and table title. |
| Outcome: | The proposed method exploits the semantic dependencies between table headers and title to extract semantic triples from tables. |
Break It Down: A Question Understanding Benchmark (2020.tacl-1)
Copied to clipboard
| Challenge: | Understanding natural language questions entails the ability to break down a question into the requisite steps for computing its answer. |
| Approach: | They introduce a Question Decomposition Meaning Representation (QDMR) for questions . they demonstrate that QDMRs can be annotated at scale using a hotpotQA dataset . |
| Outcome: | The proposed model outperforms several natural baselines in the open-domain question answering hotpotQA dataset and can be deterministically converted to a pseudo-SQL formal language. |
Injecting Numerical Reasoning Skills into Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models for numerical reasoning are limited by their flexibility and require specialized architectures to capture high-level skills. |
| Approach: | They propose to inject numerical reasoning skills into pre-trained LMs by generating large amounts of data and training in a multi-task setup. |
| Outcome: | The proposed model performs better on DROP than other models of comparable size while maintaining high performance on standard RC tasks. |
SCROLLS: Standardized CompaRison Over Long Language Sequences (2022.emnlp-main)
Copied to clipboard
Uri Shaham, Elad Segal, Maor Ivgi, Avia Efrat, Ori Yoran, Adi Haviv, Ankit Gupta, Wenhan Xiong, Mor Geva, Jonathan Berant, Omer Levy
| Challenge: | Standard NLP benchmarks focus on short texts, but long texts are produced in the context of longer discourses. |
| Approach: | They propose a new benchmark that places models in context of long texts that require reasoning over long texts. |
| Outcome: | The proposed task sets are based on a set of long-text datasets and host a live leaderboard to facilitate research on model architecture and pretraining methods. |
BioVLM: Routing Prompts, Not Parameters, for Cross-Modality Generalization in Biomedical VLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Pretrained biomedical vision–language models perform well on average but often degrade on challenging modalities. |
| Approach: | They propose a prompt-learning framework that improves cross-domain generalization without extensive backbone fine-tuning. |
| Outcome: | BioVLM learns a diverse prompt bank and introduces dynamic prompt selection . it can combine sparse few-shot evidence with rich LLM semantic priors . bioVLM achieves state-of-the-art on 11 MedMNIST+ 2D datasets based on the proposed framework . |
hinglishNorm - A Corpus of Hindi-English Code Mixed Sentences for Text Normalization (2020.coling-industry)
Copied to clipboard
| Challenge: | hinglishNorm is a human annotated corpus of Hindi-English code-mixed sentences for text normalization task. |
| Approach: | They propose to annotate sentences in Hindi-English code-mixed sentences using a human annotated normalized form. |
| Outcome: | The proposed corpus contains 13494 segments annotated for text normalization. |
Cross-Lingual Knowledge Distillation for Answer Sentence Selection in Low-Resource Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | Cross-Lingual Knowledge Distillation (CLKD) is a method to train AS2 models for low-resource languages without labeled data. |
| Approach: | They propose a method to train AS2 models for low-resource languages without labeled data . they use a translation-based WikiQA dataset and a multilingual AS2 dataset . |
| Outcome: | The proposed method outperforms or rivals fine-tuning with labeled data and machine translation and the teacher model. |
Analyzing Transformers in Embedding Space (2023.acl-long)
Copied to clipboard
| Challenge: | Understanding Transformer-based models has attracted significant attention . a zero-pass approach is feasible for some parameters, and for two-layer attention networks . |
| Approach: | They propose a theoretical framework where parameters of a trained Transformer are interpreted by projecting them into the embedding space. |
| Outcome: | The proposed framework shows that pre-trained and fine-tuned models can be interpreted in embedding space. |
Training Mixed-Domain Translation Models via Federated Learning (2022.naacl-main)
Copied to clipboard
| Challenge: | Experimental results show that neural machine translation engines built via FL can be easily adapted when an FL-based aggregation is applied to fuse different domains. |
| Approach: | They propose to use federated learning to fuse mixed-domain translation models with a centralized aggregation to improve their performance. |
| Outcome: | The proposed model can be easily adapted to a mixed-domain translation model with slight modifications in the training process and perform on par with state-of-the-art training models. |