Are Embedded Potatoes Still Vegetables? On the Limitations of WordNet Embeddings for Lexical Semantics (2023.emnlp-main)
Copied to clipboard
| Challenge: | Knowledge Base Embedding (KBE) models are widely used to encode structured information from knowledge bases, including WordNet, but the evaluation task is often focused on link prediction, ignoring their semantic capabilities. |
| Approach: | They propose to evaluate the performance of Knowledge Base Embedding (KBE) models of WordNet on link prediction and their ability to encode semantic information. |
| Outcome: | The proposed model performs poorly on two semantic tasks and two downstream tasks. |
Similar Papers
A Survey on Automatically-Constructed WordNets and their Evaluation: Lexical and Word Embedding-based Approaches (L18-1)
Copied to clipboard
| Challenge: | WordNets are lexical databases in which groups of synonyms are stored according to the semantic relationships between them. |
| Approach: | This paper describes various approaches to constructing WordNets automatically by leveraging traditional lexical resources and newer trends such as word embeddings. |
| Outcome: | The proposed methods leverage traditional lexical resources and newer trends such as word embeddings to build and evaluate WordNets. |
How to (Properly) Evaluate Cross-Lingual Word Embeddings: On Strong Baselines, Comparative Analyses, and Some Misconceptions (P19-1)
Copied to clipboard
| Challenge: | Cross-lingual word embeddings (CLEs) are used for downstream NLP tasks . CLEs are based on bilingual lexicon induction (BLI) evaluations vary greatly, hindering ability to interpret performance and properties of different CLE models. |
| Approach: | They evaluate CLE models for a large number of language pairs on bilingual lexicon induction and three downstream tasks. |
| Outcome: | The proposed model performance is based on supervised and unsupervised models on bilingual lexicon induction and three downstream tasks. |
Connecting Language and Knowledge with Heterogeneous Representations for Neural Relation Extraction (N19-1)
Copied to clipboard
| Challenge: | Knowledge Bases (KBs) require constant updating to reflect changes to the world they represent. |
| Approach: | They propose a framework that unifies learning of RE and KBE models . the framework is based on a relation extraction task that uses a KB relation to a phrase . |
| Outcome: | The proposed framework unifies learning of RE and KBE models, leading to significant improvements over the state-of-the-art RE framework. |
What’s in Your Embedding, And How It Predicts Task Performance (C18-1)
Copied to clipboard
| Challenge: | Attempts to find a single technique for general-purpose intrinsic evaluation of word embeddings have so far not been successful. |
| Approach: | They propose a method that quantifies interpretable characteristics of word vector neighborhoods and shows how they correlate with performance on 14 extrinsic and intrinsic task datasets. |
| Outcome: | The proposed approach enables multi-faceted evaluation, parameter search, and generally – a more principled, hypothesis-driven approach to development of distributional semantic representations. |
Evaluation of Domain-specific Word Embeddings using Knowledge Resources (L18-1)
Copied to clipboard
| Challenge: | Existing word embeddings capture a range of semantic relations relevant to the interpretation of lexical items, but domain-specific terms are difficult to evaluate because of a lack of statistical clues in the underlying corpus. |
| Approach: | They conduct intrinsic and extrinsic evaluations of both general and domain-specific embeddings and adapt embeddment enhancement methods to provide vector representations for infrequent and unseen terms. |
| Outcome: | The proposed model improves both in the intrinsic evaluation and extrinsic evaluation of the embedding models and their representations of infrequent and unseen terms. |
Exploring Category Structure with Contextual Language Models and Lexical Semantic Networks (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent work on word embeddings reports low correlations with human ratings . contextual language models (CLMs) have been successful in acquiring semantic and world knowledge. |
| Approach: | They propose to use BERT to probe contextual language models for predicting typicality scores. |
| Outcome: | The proposed methods improve on previous studies on word embeddings and their ability to predict typicality scores. |
A Systematic Investigation of KB-Text Embedding Alignment at Scale (2021.acl-long)
Copied to clipboard
| Challenge: | Knowledge bases (KBs) and text often contain complementary knowledge. |
| Approach: | They propose a framework for aligning KB and text embeddings for joint reasoning . they also evaluate alignment methods to infuse textual information into KB embeddables . |
| Outcome: | The proposed framework can be used to predict link prediction on emerging entities and events using textual information. |
Exploring the Value of Personalized Word Embeddings (2020.coling-main)
Copied to clipboard
| Challenge: | a subset of words belonging to specific psycholinguistic categories vary more in their representations across users . combining generic and personalized word embeddings yields the best performance . |
| Approach: | They propose personalized word embeddings and compare their performance to generic ones . they show that personalized word representations can be leveraged for improved performance . |
| Outcome: | The proposed model can be used for authorship attribution. |
Just Rank: Rethinking Evaluation with Word and Sentence Similarities (2022.acl-long)
Copied to clipboard
| Challenge: | Word and sentence similarity tasks are the de facto evaluation method for embeddings. |
| Approach: | They propose a new intrinsic evaluation method called EvalRank which shows a much stronger correlation with downstream tasks. |
| Outcome: | The proposed method shows a much stronger correlation with downstream tasks and is released for future benchmarking purposes. |
Beyond Model Performance: Can Link Prediction Enrich French Lexical Graphs? (2024.lrec-main)
Copied to clipboard
| Challenge: | lexical resources are essential for the development of NLP systems, but with advances in language models and deep learning, they are increasingly being replaced by web-derived text. |
| Approach: | They propose a resource-centric study of link prediction approaches over French lexical-semantic graphs. |
| Outcome: | The proposed method is more accurate and reliable than previous methods. |