Why does PairDiff work? - A Mathematical Analysis of Bilinear Relational Compositional Operators for Analogy Detection (C18-1)
Copied to clipboard
| Challenge: | Existing methods for representing semantic relations between words are unclear . identifying relations between word and entity is important for NLP applications . |
| Approach: | They propose to compute the vector offset between word embeddings to represent relation between two words . they show that PairDiff is an uncorrelated bilinear operator that can be simplified to a linear form . |
| Outcome: | The proposed method is surprisingly accurate and can be used on multiple word embeddings. |
Similar Papers
pair2vec: Compositional Word-Pair Embeddings for Cross-Sentence Inference (N19-1)
Copied to clipboard
| Challenge: | Existing inference models that rely heavily on unsupervised single-word embeddings struggle to learn implied relationships between pairs of words. |
| Approach: | They propose to use word embeddings to learn and use background knowledge about implied relationships between words that are crucial for cross-sentence inference problems. |
| Outcome: | The proposed models gain 2.7% on the recently released SQuAD 2.0 and 1.3% on MultiNLI, and 8.8% on the adversarial SQu AD datasets. |
RPD: A Distance Function Between Word Embeddings (2020.acl-srw)
Copied to clipboard
| Challenge: | Existing word embeddings are poorly understood, but little is known about how they differ between different sets of word embeds. |
| Approach: | They propose a metric called Relative Pairwise Inner Product Distance to quantify the distance between different word embeddings. |
| Outcome: | The proposed metric measures the distance between different sets of embeddings and investigates the influence of different training processes and corpora. |
Analyzing the Surprising Variability in Word Embedding Stability Across Languages (2021.emnlp-main)
Copied to clipboard
| Challenge: | Word embeddings are powerful representations that form the foundation of many natural language processing architectures. |
| Approach: | They explore word embedding stability in a wide range of languages to gain insight into their stability. |
| Outcome: | The proposed results provide insights into word embedding stability in English and other languages. |
Collocation Classification with Unsupervised Relation Vectors (P19-1)
Copied to clipboard
| Challenge: | Existing methods for relation classification are based on word embeddings, but they pose a greater challenge than syntactic and semantic relations. |
| Approach: | They propose a distributional landscape based on word embeddings as a suitable basis for relation classification of collocations . they also conduct experiments on a subset of this benchmark . |
| Outcome: | The proposed dataset is compared to the well known DiffVec dataset and shows that it is more efficient than the standard methods. |
From Text to Lexicon: Bridging the Gap between Word Embeddings and Lexical Resources (C18-1)
Copied to clipboard
| Challenge: | Distributional word representations are omnipresent in modern NLP. |
| Approach: | They propose to combine lemmatization and part of speech (POS) typing to improve word embedding performance. |
| Outcome: | The proposed methods improve word embedding performance on verbs and verbs. |
The Word Analogy Testing Caveat (N18-2)
Copied to clipboard
| Challenge: | a number of word analogy tests are used to evaluate word embeddings . word embeds are used as a proxy for semantics and syntax à la Harris . |
| Approach: | They propose to use word embeddings as a proxy for distributional similarity . they propose to apply a transfer learning approach to word embeds to improve performance . |
| Outcome: | The proposed method improves performance across a wide range of NLP tasks. |
Enhanced Word Representations for Bridging Anaphora Resolution (N18-2)
Copied to clipboard
| Challenge: | Existing word representations do not capture semantic similarity for bridging anaphora resolution. |
| Approach: | They propose to use word embeddings to capture semantic similarity by exploring syntactic structure of noun phrases. |
| Outcome: | The proposed model achieves 30% of accuracy for bridging anaphora resolution on ISNotes corpus. |
PairRE: Knowledge Graph Embeddings via Paired Relation Vectors (2021.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge graph embedding methods fail to solve two major problems at the same time, leading to unsatisfactory results. |
| Approach: | They propose a model with paired vectors for each relation representation that can be adaptively adjusted to fit for different complex relations. |
| Outcome: | Experiments on two knowledge graph datasets show the proposed model can handle complex relations and encode relation patterns. |
Paraphrases do not explain word analogies (2021.eacl-main)
Copied to clipboard
| Challenge: | Several attempts have been made to explain distributional word embeddings as linguistic regularities as directions. |
| Approach: | They propose to use an analogy to explain why linguistic regularities should hold in distributional word embeddings. |
| Outcome: | The proposed explanation does not hold empirically. |
Towards Understanding Linear Word Analogies (P19-1)
Copied to clipboard
| Challenge: | Existing theories of word embeddings make strong assumptions about the embeddable space or word distribution. |
| Approach: | They propose a formal explanation of word analogies by adding arithmetic operators to non-linear embedding models such as skip-gram with negative sampling. |
| Outcome: | The proposed model downweights the more frequent word, as weighting schemes do ad hoc. |