Non-Autoregressive Semantic Parsing for Compositional Task-Oriented Dialog (2021.naacl-main)
Copied to clipboard
| Challenge: | Semantic parsing using sequence-to-sequence models is stymied by higher compute requirements and higher latency. |
| Approach: | They propose a non-autoregressive approach to predict semantic parse trees with an efficient seq2seq model architecture. |
| Outcome: | The proposed architecture achieves an 81% reduction in latency on TOP dataset and retains competitive performance over non-pretrained models on three different semantic parsing datasets. |
Similar Papers
Span Pointer Networks for Non-Autoregressive Task-Oriented Semantic Parsing (2021.findings-emnlp)
Copied to clipboard
Akshat Shrivastava, Pierce Chuang, Arun Babu, Shrey Desai, Abhinav Arora, Alexander Zotov, Ahmed Aly
| Challenge: | a novel approach to map utterances to semantic frames is based on non-autoregressive parsers that shift the decoding task from text generation to span prediction. |
| Approach: | They propose a non-autoregressive, task-oriented parser which shifts the decoding task from text generation to span prediction and produces endpoints as opposed to text. |
| Outcome: | The proposed model bridges the quality gap between non-autoregressive and autoregressive parsers, achieving 87 EM on TOPv2 and shows a 70% reduction in latency and 83% reduction in memory at beam size 5 compared to prior non-regressives. |
Compositional Generalization without Trees using Multiset Tagging and Latent Permutations (2023.acl-long)
Copied to clipboard
| Challenge: | Seq2seq models struggle with compositional generalization in semantic parsing, i.e. generalizing to unseen compositions or deeper recursion of phenomena that the model handles correctly in isolation. |
| Approach: | They propose a new way of parameterizing and predicting permutations by combining input tokens with multisets of output tokens and a method to backpropagate through the solver. |
| Outcome: | The proposed model outperforms pretrained models and prior work on realistic semantic parsing tasks that require generalization to longer examples. |
Tree-Structured Non-Autoregressive Decoding for Sequence-to-Sequence Text Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Autoregressive Transformers suffer from high inference latency due to sequential token generation. |
| Approach: | They propose a tree-structured non-autoregressive decoding paradigm that bridges autoregressive and non-automatic decoding. |
| Outcome: | The proposed paradigm outperforms autoregressive and non-autoregressive decoding in machine translation and paraphrase generation. |
Low-Resource Compositional Semantic Parsing with Concept Pretraining (2023.eacl-main)
Copied to clipboard
| Challenge: | Semantic parsing is a key role in voice assistants by mapping natural language to structured meaning representations. |
| Approach: | They propose an architecture to perform domain adaptation automatically with only a small amount of metadata about the new domain and without any new training data. |
| Outcome: | The proposed architecture outperforms existing models in low-resource settings. |
Fully Non-autoregressive Neural Machine Translation: Tricks of the Trade (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing non-autoregressive neural machine translation models are slow to learn the dependency between output tokens. |
| Approach: | They propose to use fully non-autoregressive neural machine translation (NAT) to predict tokens with single forward of neural networks. |
| Outcome: | The proposed model achieves state-of-the-art results on three translation benchmarks with comparable performance to autoregressive and iterative NAT systems. |
Improving AMR Parsing with Sequence-to-Sequence Pre-training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Abstract meaning representation (AMR) parsing is limited by the size of curated datasets. |
| Approach: | They propose a seq2seq pre-training approach to build pre-trained models on three relevant tasks. |
| Outcome: | The proposed model improves performance on three relevant tasks while maintaining the response of pre-trained models. |
Non-Autoregressive Neural Machine Translation: A Call for Clarity (2022.emnlp-main)
Copied to clipboard
| Challenge: | Non-autoregressive translation models require a single forward pass to generate the output sequence instead of iteratively producing each predicted token. |
| Approach: | They propose to use a single forward pass to generate the output sequence instead of iteratively producing each predicted token. |
| Outcome: | The proposed models improve translation quality and speed under third-party testing environments. |
Non-Autoregressive Models for Fast Sequence Generation (2022.emnlp-tutorials)
Copied to clipboard
| Challenge: | Autoregressive (AR) models can only generate target sequence word-by-word due to the AR mechanism and suffer from slow inference. |
| Approach: | This tutorial provides an introduction to non-autoregressive sequence generation. |
| Outcome: | This tutorial explains how to generate non-autoregressive sequence generation models. |
Compositional Generalization via Semantic Tagging (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing neural sequence-to-sequence models fail at compositional generalization, i.e., they cannot generalize to unseen compositions of seen components. |
| Approach: | They propose a decoding framework that preserves expressivity and generality of sequence-to-sequence models while featuring lexicon-style alignments and disentangled information processing. |
| Outcome: | The proposed framework improves compositional generalization across model architectures, domains, and semantic formalisms on three semantic parsing datasets. |
End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classification (D18-1)
Copied to clipboard
| Challenge: | Autoregressive decoding is the only part of sequence-to-sequence models that prevents massive parallelization at inference time. |
| Approach: | They propose a non-autoregressive architecture based on connectionist temporal classification . they conduct experiments on the WMT English-Romanian and English-German datasets . |
| Outcome: | The proposed model achieves a significant speedup over autoregressive models . the model can be trained end-to-end and maintains translation quality comparable to other models compared to autoregression models based on connectionist temporal classification . |