Coming to Your Senses: on Controls and Evaluation Sets in Polysemy Research (D18-1)
Copied to clipboard
| Challenge: | a prior art claim that sense-specific vectors provide an advantage over normal vectors is unfounded in two ways. |
| Approach: | They claim that sense-specific vectors provide an advantage over normal vectors due to the polysemy that they presumably represent. |
| Outcome: | The proposed results show that ground-truth polysemy degrades performance in word similarity tasks and that random assignment of words to senses improves performance. |
Similar Papers
Which Evaluations Uncover Sense Representations that Actually Make Sense? (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing sense representations fail for human-centric tasks like inspecting a language’s sense inventory. |
| Approach: | They propose a coherence evaluation for sense embeddings and a model optimized for finding interpretable sense representations that are more coherent than existing sense embeds. |
| Outcome: | The proposed model is more coherent than existing sense embeddings and offers comparable word similarities with multisense representations while learning more distinguishable, interpretable senses. |
PolyLM: Learning about Polysemy through Language Modeling (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods to embed word senses have been overtaken by contextualized embeddings . alan ansell and jim koenig present a method which can be applied to downstream tasks . |
| Approach: | They propose a method which formulates learning sense embeddings as a language modeling problem. |
| Outcome: | The proposed method performs better than existing sense embedding methods on WSI tasks . it matches the current state-of-the-art specialized WSi method despite having six times fewer parameters . |
MSD-1030: A Well-built Multi-Sense Evaluation Dataset for Sense Representation Models (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing benchmarks for sense embeddings do not account for polysemy, and there are six issues with evaluations based on these datasets. |
| Approach: | They propose a multi-sense dataset with a high ratio of multi-word pairs to address the polysemy issue in word embeddings. |
| Outcome: | The proposed model performs better than existing models with single-sense word pairs and has a high ratio of multi-sensor word pairs. |
What just happened? Evaluating retrofitted distributional word vectors (N19-1)
Copied to clipboard
| Challenge: | Recent work has attempted to enhance vector space representations using information from structured semantic resources. |
| Approach: | They propose a root-mean-square error evaluation metric to evaluate the utility of different lexical resources for retrofitting. |
| Outcome: | The proposed method improves word similarity performance by using root-mean-square error (RMSE) and root-macro-error (RMME) metric. |
Retrofitting Word Representations for Unsupervised Sense Aware Word Similarities (L18-1)
Copied to clipboard
| Challenge: | Standard word embeddings lack the ability to distinguish senses of a word by projecting them to exactly one vector. |
| Approach: | They propose to retrofit standard word embeddings to produce sense-aware embeddable vectors using external resources as sense inventories. |
| Outcome: | The proposed method improves word similarity and relatedness scores on multiple word embeddings and established word similarities, sometimes up to an impressive margin of +0.15 Spearman correlation score. |
On the Importance of Distinguishing Word Meaning Representations: A Case Study on Reverse Dictionary Mapping (N19-1)
Copied to clipboard
| Challenge: | Sense representations target meaning conflation deficiency but their potential impact has not been investigated in downstream NLP applications. |
| Approach: | They propose to use a reverse dictionary system to address meaning conflation deficiency . they propose to integrate senses into the system to improve semantic understanding . |
| Outcome: | The proposed approach can improve the performance of a downstream NLP application. |
SensePOLAR: Word sense aware interpretability for pre-trained contextual word embeddings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing word embedding models lack interpretability for words . |
| Approach: | They propose to add interpretability to word embeddings by using a POLAR framework that enables wordsense aware interpretations for pre-trained contextual word embeds. |
| Outcome: | The proposed framework achieves comparable performance to existing embeddings across GLUE and SQuAD benchmarks. |
What’s in Your Embedding, And How It Predicts Task Performance (C18-1)
Copied to clipboard
| Challenge: | Attempts to find a single technique for general-purpose intrinsic evaluation of word embeddings have so far not been successful. |
| Approach: | They propose a method that quantifies interpretable characteristics of word vector neighborhoods and shows how they correlate with performance on 14 extrinsic and intrinsic task datasets. |
| Outcome: | The proposed approach enables multi-faceted evaluation, parameter search, and generally – a more principled, hypothesis-driven approach to development of distributional semantic representations. |
Unsupervised Word Polysemy Quantification with Multiresolution Grids of Contextual Embeddings (2021.eacl-main)
Copied to clipboard
| Challenge: | a new method to quantify polysemy is based on basic geometry in the contextual embedding space . word sense annotation has always been one of the tasks with the lowest interannotator agreement . |
| Approach: | They propose a method to estimate polysemy based on simple geometry in contextual embedding space. |
| Outcome: | The proposed method is fully unsupervised and data-driven . it can be used to sample sentences with different senses at no extra cost . |
Advances in Pre-Training Distributed Word Representations (L18-1)
Copied to clipboard
| Challenge: | Pre-trained word representations are a building block of many Natural Language Processing and Machine Learning applications. |
| Approach: | They propose to combine known tricks and a set of publicly available pre-trained word vector representations to train high-quality representations. |
| Outcome: | The proposed models outperform the current state of the art on a number of tasks while maintaining a high training speed to scale to massive amount of data. |