Transferring Knowledge from Structure-aware Self-attention Language Model to Sequence-to-Sequence Semantic Parsing (2022.coling-1)
Copied to clipboard
| Challenge: | Semantic parsing aims to map a natural language sentence into a machine executable formal representation. |
| Approach: | They propose a structure-aware self-attention language model to capture structural information of target representations and propose incorporating it into a seq2seq model. |
| Outcome: | The proposed model improves the baseline model on four semantic parsing and Python code generation tasks. |
Similar Papers
Grammar-Constrained Neural Semantic Parsing with LR Parsers (2021.findings-acl)
Copied to clipboard
| Challenge: | a context-free grammar can be used to enforce syntactical constraints when predicting logical forms. |
| Approach: | They propose a model that uses an LR parser to maintain syntactically valid sequences throughout the decoding procedure. |
| Outcome: | The proposed model is conceptually simpler and adds less overhead during inference compared to other approaches . it is compared with existing grammar-guided decoding frameworks and is cost-effective . |
DRTS Parsing with Structure-Aware Encoding and Decoding (2020.acl-main)
Copied to clipboard
| Challenge: | Discourse representation tree structure (DRTS) parsing is a new semantic parser which ignores structural information. |
| Approach: | They propose a structural-aware model to integrate structural information into the model . they use graph attention network (GAT) to exploit structural information for effective modeling . |
| Outcome: | The proposed model can achieve the best performance on a benchmark dataset. |
Leveraging AMR Graph Structure for Better Sequence-to-Sequence AMR Parsing (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent studies on AMR parsing often regard this task as a seq2seq translation problem. |
| Approach: | They propose to translate AMR graphs into AMR token sequences in pre-processing and recover AMR from sequences after decoding. |
| Outcome: | The proposed approach outperforms baseline and achieves 85.5 0.1 and 84.2 0.2 Smatch scores on AMR 2.0 and AMR 3.0. |
Modeling Graph Structure in Transformer for Better AMR-to-Text Generation (D19-1)
Copied to clipboard
| Challenge: | Recent studies on AMR-to-text generation formalize the task as a sequence-tosequence learning problem . previous approaches only consider the relations between directly connected concepts while ignoring the rich structure in AMR graphs. |
| Approach: | They propose a structure-aware self-attention approach to model the relations between indirectly connected concepts in the seq2seq model. |
| Outcome: | The proposed approach outperforms the state-of-the-art on English AMR benchmarks . it significantly outperformed the state of the art on the benchmarks, with 29.66 and 31.82 BLEU scores . |
Coarse-to-Fine Decoding for Neural Semantic Parsing (P18-1)
Copied to clipboard
| Challenge: | Experimental results show that semantic parsing is more efficient than using simple decoders. |
| Approach: | They propose a structure-aware neural architecture which decomposes the semantic parsing process into two stages. |
| Outcome: | The proposed architecture consistently improves performance on four datasets characteristic of different domains and meaning representations. |
Top-down Tree Structured Decoding with Syntactic Connections for Neural Machine Translation and Parsing (D18-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) models are based on sequential decoding or serialisation of structured data into sequence. |
| Approach: | They propose a model that combines sequential encoder with tree-structured decoding augmented with a syntax-aware attention model. |
| Outcome: | The proposed model produces fluent translations with better reordering than previous models. |
Improving AMR Parsing with Sequence-to-Sequence Pre-training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Abstract meaning representation (AMR) parsing is limited by the size of curated datasets. |
| Approach: | They propose a seq2seq pre-training approach to build pre-trained models on three relevant tasks. |
| Outcome: | The proposed model improves performance on three relevant tasks while maintaining the response of pre-trained models. |
Scalable Syntax-Aware Language Models Using Knowledge Distillation (P19-1)
Copied to clipboard
| Challenge: | Prior work has shown that syntactic neural language models learn from small amounts of training data more effectively than sequential models. |
| Approach: | They propose a knowledge distillation technique that transfers knowledge from a syntactic language model trained on a small corpus to an LSTM language model and enables it to develop a more structurally sensitive representation of the larger training data. |
| Outcome: | The proposed method improves on baseline syntactic evaluations on LSTMs with a higher level of accuracy than previous methods. |
Linguistically-Informed Self-Attention for Semantic Role Labeling (D18-1)
Copied to clipboard
| Challenge: | Existing models of semantic role labeling use no explicit linguistic features. prior work has shown that syntax trees can dramatically improve SRL decoding. |
| Approach: | They propose a neural network model that incorporates syntax using only raw tokens . they show that LISA out-performs the state-of-the-art with contextually-encoded word representations a 1.0 F1 on newswire and 2.0 F1 in out-of domain text . |
| Outcome: | The proposed model outperforms the state-of-the-art model with word embeddings and predicted predicates. |
Enhancing Machine Translation with Dependency-Aware Self-Attention (2020.acl-main)
Copied to clipboard
| Challenge: | Currently, most neural machine translation models rely on pairs of parallel sentences, assuming syntactic information is automatically learned by an attention mechanism. |
| Approach: | They propose a parameter-free, dependency-aware self-attention mechanism that integrates syntactic knowledge into a Transformer model and propose 'a parameter free approach' they also propose - a novel mechanism that improves translation quality for long sentences and in low-resource scenarios. |
| Outcome: | The proposed approach improves translation quality on English-German and English-Turkish translation tasks and in low-resource scenarios. |