Improved Dependency Parsing using Implicit Word Connections Learned from Unlabeled Data (D18-1)
Copied to clipboard
| Challenge: | Pre-trained word embeddings and language models cannot capture word connections in a sentence. |
| Approach: | They propose to implicitly capture word connections from unlabeled data by word ordering model with self-attention mechanism. |
| Outcome: | The proposed model achieves 96.35% UAS and 95.25% LAS on the English PTB dataset. |
Similar Papers
Improving Relation Extraction through Syntax-induced Pre-training with Dependency Masking (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing studies require modifications to existing baseline architectures to leverage syntactic information. |
| Approach: | They propose to leverage syntactic information to improve relation extraction by training a syntax-induced encoder on auto-parsed data through dependency masking. |
| Outcome: | The proposed approach outperforms baseline models and achieves state-of-the-art results on two English datasets. |
Modal Dependency Parsing via Language Model Priming (2022.naacl-main)
Copied to clipboard
| Challenge: | modal dependency parsing is a task of parse a text into its modal dependence structure . the root node of an MDS is always the author of a document, the ultimate source of information sources . |
| Approach: | They propose a modal dependency parser based on priming pre-trained language models and evaluate it on two data sets. |
| Outcome: | The proposed parser improves on two data sets. |
Enhancing Structure-aware Encoder with Extremely Limited Data for Graph-based Dependency Parsing (2022.coling-1)
Copied to clipboard
| Challenge: | Dependency parsing is an important natural language processing task which analyzes the syntactic structure of an input sentence. |
| Approach: | They propose a structure-aware encoder pre-trained on auto-parsed data to improve dependency parsing . they propose combining gold dependency trees with existing parsers to improve parser performance . |
| Outcome: | The proposed approach outperforms baselines under different parsers and dependency standards under different parameters and model architectures. |
Dependency Position Encoding for Relation Extraction (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to extract relation extraction from sentence are limited in focusing on leveraging dependency information. |
| Approach: | They propose dependency position encoding (DPE) that incorporates dependency connections and dependency types into the self-attention mechanism to distinguish the importance of different word dependencies. |
| Outcome: | The proposed method significantly outperforms the previous methods on SemEval 2010 Task 8, KBP37, and TACRED. |
Improving Relation Extraction with Knowledge-attention (D19-1)
Copied to clipboard
| Challenge: | Existing attention mechanisms are data-driven, but most are data driven. |
| Approach: | They propose a knowledge-attention encoder which integrates prior knowledge from external lexical resources into deep neural networks for relation extraction task. |
| Outcome: | The proposed system outperforms existing CNN, RNN, and self-attention based models on a large-scale relation extraction dataset. |
Improving Semantic Matching through Dependency-Enhanced Pre-trained Model with Adaptive Fusion (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on dependency prior structure integration into pre-trained models is still unclear. |
| Approach: | They propose a dependency-based fusion attention paradigm which explicitly introduces dependency prior structure into pre-trained models and adaptively fuses it with semantic information. |
| Outcome: | The proposed model achieves state-of-the-art or competitive performance on 10 public datasets, demonstrating the benefits of adaptively fusing dependency structure in semantic matching task. |
How to represent a word and predict it, too: Improving tied architectures for language modelling (D18-1)
Copied to clipboard
| Challenge: | Recent state-of-the-art models use word embeddings as input and output mappings instead of tied models. |
| Approach: | They propose to decouple hidden state from word embedding prediction . they extend their proposed modification to word2vec models . |
| Outcome: | The proposed architectures achieve comparable or better results compared to previous models without tying . the proposed architecture reduces parameters, enabling more compact models and faster learning. |
Exploiting the Syntax-Model Consistency for Neural Relation Extraction (2020.acl-main)
Copied to clipboard
| Challenge: | Existing deep learning models for Relation Extraction (RE) have limited generalization beyond the syntactic structures of the input sentences. |
| Approach: | They propose a deep learning model that uses dependency trees to extract syntactic importance of words for Relation Extraction. |
| Outcome: | The proposed model outperforms existing models on three RE benchmark datasets. |
Simpler but More Accurate Semantic Dependency Parsing (P18-2)
Copied to clipboard
| Challenge: | Syntactic dependency parsing is the most popular method for automatically extracting low-level relationships between words in a sentence. |
| Approach: | They extend a syntactic dependency parser to train on and generate graph-structured representations that capture between-word relationships that are more closely related to the meaning of a sentence. |
| Outcome: | The proposed system beats the current state-of-the-art system by 0.6% and linguistically richer representations push the margin even higher. |
Improving Relation Extraction by Sequence-to-sequence-based Dependency Parsing Pre-training (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies show that dependency information is used only for encoder-only-based relation extraction tasks. |
| Approach: | They propose a syntax-aware seq2seq pre-trained model for relation extraction that incorporates dependency information into a seq2-trained language model by continual pre-training with a dependency parsing task. |
| Outcome: | The proposed model incorporates dependency information into a seq2seq pre-trained language model by continual pre-training with a generative sequence-to-sequence (sequ2sq)-based dependency parsing task. |