Incorporating Syntactic and Semantic Information in Word Embeddings using Graph Convolutional Networks (P19-1)
Copied to clipboard
Shikhar Vashishth, Manik Bhandari, Prateek Yadav, Piyush Rai, Chiranjib Bhattacharyya, Partha Talukdar
| Challenge: | Existing word embedding methods utilize sequential context of a word to learn its embeddment, but such methods result in an explosion of the vocabulary size. |
| Approach: | They propose a flexible Graph Convolution based method for learning word embeddings that utilizes the dependency context of a word without increasing the vocabulary size. |
| Outcome: | The proposed model outperforms existing methods on intrinsic and extrinsic tasks and provides an advantage when used with ELMo. |
Similar Papers
ConTextING: Granting Document-Wise Contextual Embeddings to Graph Neural Networks for Inductive Text Classification (2022.coling-1)
Copied to clipboard
| Challenge: | Graph neural networks (GNNs) are used to learn document representation from graph structures. |
| Approach: | They propose a unified model with a joint training mechanism to learn from document embeddings and contextual word interactions simultaneously. |
| Outcome: | The proposed model outperforms pure inductive GNNs and BERT-style models . the proposed model also has a joint training mechanism to learn from document embeddings and contextual word interactions simultaneously. |
Using Context-to-Vector with Graph Retrofitting to Improve Word Embeddings (2022.acl-long)
Copied to clipboard
| Challenge: | Contextualized embeddings are expensive and resource-demanding, hence environmentally unfriendly. |
| Approach: | They propose a method to convert contextualized embeddings from pre-trained models into static embeddables using synonym knowledge and weighted vector distribution. |
| Outcome: | The proposed method outperforms baseline embeddings by a large margin through extrinsic and intrinsic tasks. |
Improved Semantic-Aware Network Embedding with Fine-Grained Word Alignment (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to network embeddings focus on one-hot representations of vertices, which are not able to capture relationships between verti- ces. |
| Approach: | They propose to integrate semantic features into network embeddings by matching important words between text sequences for all pairs of vertices. |
| Outcome: | The proposed framework outperforms state-of-the-art embedding methods on three real-world benchmarks for downstream tasks including link prediction and multi-label vertex classification. |
Embedding Words in Non-Vector Space with Unsupervised Graph Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | GraphGlove is an unsupervised graph word representations that are learned end-to-end. |
| Approach: | They propose a method to learn weighted graph word representations end-to-end using a weighteable weighte . they adopt a hierarchical graph representation method and modify the GloVe training algorithm to learn graph representations. |
| Outcome: | The proposed method outperforms vector-based methods on word similarity and analogy tasks. |
Embeddings in Natural Language Processing (2020.coling-tutorials)
Copied to clipboard
| Challenge: | Embeddings have been a key topic of interest in NLP for the past decade . a quick warm-up introduction to NLP and why it is important to have a semantic comprehension of texts . |
| Approach: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and then move to other types of embeddable vectors . |
| Outcome: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and move to other types of embeddable representations . |
Using Graphs for Word Embedding with Enhanced Semantic Relations (D19-53)
Copied to clipboard
| Challenge: | Word embedding algorithms are based on sequential text input, while others are utilizing a graph representation of text. |
| Approach: | They propose a word embedding algorithm based on a directed word graph to provide additional information for sequential text input algorithms. |
| Outcome: | The proposed algorithm is based on a directed word graph to provide additional information for sequential text input algorithms. |
Semantic Frame Induction using Masked Word Embeddings and Two-Step Clustering (2021.acl-short)
Copied to clipboard
| Challenge: | Recent studies show that clustering-based methods focus too much on the surface information of frame-evoking verbs and divide instances of the same verb into too many different frame clusters. |
| Approach: | They propose a semantic frame induction method using masked word embeddings and two-step clustering to overcome these drawbacks. |
| Outcome: | The proposed method reduces the number of instances of the same verb into too many clusters . it uses masked word embeddings and two-step clustering to avoid drawbacks compared with other methods . |
Graph Convolutions over Constituent Trees for Syntax-Aware Semantic Role Labeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Semantic role labeling (SRL) is the task of identifying predicates and labeling argument spans with semantic roles. |
| Approach: | They propose to use graph convolutional networks to encode constituents and inform an SRL system by combining word representations of the first and last words in a constituent tree. |
| Outcome: | The proposed model is compared with other models and shows that it is more efficient than dependency trees. |
Enhancing Word Embeddings with Knowledge Extracted from Lexical Resources (2020.acl-srw)
Copied to clipboard
| Challenge: | In this paper, we present an effective method for semantic specialization of word vector representations. |
| Approach: | They propose a method for semantic specialization of word vector representations using BabelNet. |
| Outcome: | The proposed method improves on word similarity and dialog state tracking tasks. |
DynGL-SDP: Dynamic Graph Learning for Semantic Dependency Parsing (2022.coling-1)
Copied to clipboard
| Challenge: | Existing parsers that learn graph representations based on static graphs are error-prone and disjointed . Graph-based parser can parse sentences efficiently but suffer from error propagation . |
| Approach: | They propose a dynamic graph learning framework to learn graph representations based on a static graph constructed by an existing parser. |
| Outcome: | The proposed parser outperforms the previous parsers on the SemEval-2015 task 18 dataset in three languages. |