Post-Specialisation: Retrofitting Vectors of Words Unseen in Lexical Resources (N18-1)
Copied to clipboard
| Challenge: | Word vector specialisation is a portable, light-weight approach to fine-tuning distributional word vector spaces by injecting external knowledge from rich lexical resources such as WordNet. |
| Approach: | They propose a constraint-driven vector space specialisation method that embeds external knowledge into lexical resources into a deep neural network to specialise unseen words. |
| Outcome: | The proposed method preserves useful linguistic knowledge for seen words while propagating external signal to unseen words to improve their vector representations. |
Similar Papers
Explicit Retrofitting of Distributional Word Vectors (P18-1)
Copied to clipboard
| Challenge: | Existing models for word vector specialization focus on word co-occurrences from large text corpora, resulting in a tendency to fuse semantic similarity with other types of semantic relatedness. |
| Approach: | They propose to transform external lexico-semantic relations into training examples and learn an explicit retrofitting model. |
| Outcome: | The proposed model can specialize vector spaces of new languages and translate them to other languages. |
Self-supervised Post-processing Method to Enrich Pretrained Word Vectors (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods that use external resources to make word vectors specialize depend on the lexicon. |
| Approach: | They propose a self-supervised extension of extrofitting by its own word vector distribution. |
| Outcome: | The proposed method improves word similarity embeddings on similarity tasks without external resources. |
RetroGAN: A Cyclic Post-Specialization System for Improving Out-of-Knowledge and Rare Word Representations (2021.findings-acl)
Copied to clipboard
| Challenge: | Retrofitting is a technique used to move word vectors closer together or further apart in their space to reflect their relationships in a Knowledge Base (KB). |
| Approach: | They propose a system that uses two GANs to learn a one-to-one mapping between concepts and retrofitted counterparts. |
| Outcome: | The proposed system performs well on word-similarity benchmarks and a sentence simplification task. |
Using Context-to-Vector with Graph Retrofitting to Improve Word Embeddings (2022.acl-long)
Copied to clipboard
| Challenge: | Contextualized embeddings are expensive and resource-demanding, hence environmentally unfriendly. |
| Approach: | They propose a method to convert contextualized embeddings from pre-trained models into static embeddables using synonym knowledge and weighted vector distribution. |
| Outcome: | The proposed method outperforms baseline embeddings by a large margin through extrinsic and intrinsic tasks. |
Semantic Specialization of Distributional Word Vectors (D19-2)
Copied to clipboard
| Challenge: | Distributional word vectors conflate various paradigmatic and syntagmatic lexico-semantic relations. |
| Approach: | This tutorial provides an overview of specialization methods for distributional word vectors . a common solution is to include external lexico-semantic knowledge in a reshaped vector space . |
| Outcome: | This paper provides an overview of specialization methods for distributional word vectors . the most recent developments include a new method for asymmetric relations in Euclidean . |
Adversarial Propagation and Zero-Shot Cross-Lingual Transfer of Word Vector Specialization (D18-1)
Copied to clipboard
| Challenge: | Semantic specialization is a process of fine-tuning pre-trained distributional word vectors using external lexical knowledge to accentuate a particular semantic relation in the specialized vector space. |
| Approach: | They propose a method for specializing distributional word vectors using external lexical knowledge. |
| Outcome: | The proposed method improves on word similarity, dialog state tracking, and lexical simplification across three languages and on three tasks. |
What just happened? Evaluating retrofitted distributional word vectors (N19-1)
Copied to clipboard
| Challenge: | Recent work has attempted to enhance vector space representations using information from structured semantic resources. |
| Approach: | They propose a root-mean-square error evaluation metric to evaluate the utility of different lexical resources for retrofitting. |
| Outcome: | The proposed method improves word similarity performance by using root-mean-square error (RMSE) and root-macro-error (RMME) metric. |
Enhancing Word Embeddings with Knowledge Extracted from Lexical Resources (2020.acl-srw)
Copied to clipboard
| Challenge: | In this paper, we present an effective method for semantic specialization of word vector representations. |
| Approach: | They propose a method for semantic specialization of word vector representations using BabelNet. |
| Outcome: | The proposed method improves on word similarity and dialog state tracking tasks. |
Subword Attention and Post-Processing for Rare and Unknown Contextualized Embeddings (2024.findings-naacl)
Copied to clipboard
| Challenge: | Word embeddings are useful, but struggle on rare and unknown words. |
| Approach: | They propose a rare/unknown embedding architecture that focuses on contextualized representations. |
| Outcome: | The proposed architecture improves performance in most intrinsic and downstream tasks. |
Cross-lingual Semantic Specialization via Lexical Relation Induction (D19-1)
Copied to clipboard
| Challenge: | Semantic specialization is not available in many languages because of their incomplete or non-existent structure. |
| Approach: | They propose a method that transfers specialization from a resource-rich source language to virtually any target language. |
| Outcome: | The proposed method performs lexical simplification, dialog state tracking, and textual similarity tasks in 5 languages. |