Papers by Anchit Gupta
Retrieve-and-Fill for Scenario-based Task-Oriented Semantic Parsing (2023.eacl-main)
Copied to clipboard
Akshat Shrivastava, Shrey Desai, Anchit Gupta, Ali Elkahky, Aleksandr Livshits, Alexander Zotov, Ahmed Aly
| Challenge: | Task-oriented semantic parsing models have achieved strong results in recent years, but they often face obstacles adapting to novel settings with distinct semantics and scarce data. |
| Approach: | They propose a scenario-based semantic parsing model which isolates coarse-grained and fine-grounded aspects of the task and solves them with off-the-shelf neural modules. |
| Outcome: | The proposed model outperforms previous approaches in high-resource, low-resourced, and multilingual settings, and is modular, differentiable, interpretable, and allows extra supervision from scenarios. |
Muppet: Massive Multi-task Representations with Pre-Finetuning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows gains from pre-training and fine-tuning that are multi-task . but it can be difficult to know which intermediate tasks will best transfer . |
| Approach: | They propose a large-scale learning stage for pre-finetuning between pre-training and fine-tun. |
| Outcome: | The proposed model improves performance on pretrained discriminators and generation models on a wide range of tasks while improving sample efficiency during fine-tuning. |
MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmark (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing datasets for task-oriented dialog systems are limited and expensive . current models are based on the simple intent and slot detection paradigm for non-compositional queries. |
| Approach: | They propose to use a multilingual dataset to scale semantic parsing models to new languages . they demonstrate an average improvement of +6.3 points on Slot F1 for existing datasets . |
| Outcome: | The proposed model achieves an average improvement of +6.3 points on Slot F1 over existing models. |
Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One? (2022.findings-emnlp)
Copied to clipboard
Xilun Chen, Kushal Lakhotia, Barlas Oguz, Anchit Gupta, Patrick Lewis, Stan Peshterliev, Yashar Mehdad, Sonal Gupta, Wen-tau Yih
| Challenge: | Existing sparse retrievers lack the ability to match salient phrases and rare entities in the query. |
| Approach: | They introduce a dense Lexical Model that can be trained to imitate a sparse one. |
| Outcome: | The proposed model outperforms sparse retrievers on a range of tasks including five question answering datasets and the MS MARCO passage retrieval. |
Adapting Pretrained Text-to-Text Models for Long Text Sequences (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing short-context models are limited in their domain coverage and can be used for long-sequence inputs. |
| Approach: | They propose to replace full attention in transformers with pooling-augmented blockwise attention and pretrain the model with a masked-span prediction task with spans of varying lengths. |
| Outcome: | The proposed model outperforms existing models on long-sequence summarization tasks and achieves competitive performance on long document corpora. |
A Study on the Efficiency and Generalization of Light Hybrid Retrievers (2023.acl-short)
Copied to clipboard
Man Luo, Shashank Jain, Anchit Gupta, Arash Einolghozati, Barlas Oguz, Debojeet Chatterjee, Xilun Chen, Chitta Baral, Peyman Heidari
| Challenge: | Recent research focuses on building neural retrievers which learn dense embeddings of query and document into a semantic space. |
| Approach: | They propose to use an indexing-efficient dense retriever to reduce hybrid retrievers' memory by using the state-based indexing algorithm. |
| Outcome: | The proposed hybrid retriever saves 13 memory while maintaining 98.0% performance on out-of-domain datasets and adversarial attacks datasets. |
Simple Local Attentions Remain Competitive for Long-Context Tasks (2022.naacl-main)
Copied to clipboard
Wenhan Xiong, Barlas Oguz, Anchit Gupta, Xilun Chen, Diana Liskovich, Omer Levy, Scott Yih, Yashar Mehdad
| Challenge: | Existing models for NLP tasks require long text sequences beyond the length limit of pretrained models. |
| Approach: | They propose to pretrain large-size NLP models using the same long-doc corpus and fine tune them for real-world long-context tasks. |
| Outcome: | The proposed models can perform better under standard pretraining paradigms than longformer and Longformer. |
Sound Natural: Content Rephrasing in Dialog Systems (2020.emnlp-main)
Copied to clipboard
| Challenge: | Currently, virtual assistants work in the paradigm of intent-slot tagging and the slot values are directly passed as-is to the execution engine. |
| Approach: | They propose to use BART to rephrase a query to make it more natural . they propose to add a copy-pointer and copy loss to it to improve performance . |
| Outcome: | The proposed model improves on existing models by adding a copy-pointer and copy loss. |
Domain-matched Pre-training Tasks for Dense Retrieval (2022.findings-naacl)
Copied to clipboard
Barlas Oguz, Kushal Lakhotia, Anchit Gupta, Patrick Lewis, Vladimir Karpukhin, Aleksandra Piktus, Xilun Chen, Sebastian Riedel, Scott Yih, Sonal Gupta, Yashar Mehdad
| Challenge: | Existing approaches to improve performance of pre-training tasks are needed. |
| Approach: | They propose to pre-train large bi-encoder models on a recently released set of 65 millionsynthetically generated questions and 200 million post-comment pairs from a preexisting reddit conversation dataset. |
| Outcome: | The proposed model can be pre-trained on a set of 65 millionsynthetically generated questions and 200 million post-comment pairs from a preexisting dataset of Reddit conversations. |