| Challenge: | l2 norm of sense embeddings encodes information related to frequency of that sense in the training corpus . l2-normal feature is useful for word-in-context (WiC) and word sense disambiguation (WSD) |
| Approach: | They propose to include the l2 norm of a sense embedding as a feature in a classifier to improve word sense learning methods that use static sense embeds. |
| Outcome: | The l2 norm of sense embeddings is a surprisingly effective feature for word sense related tasks such as word-in-context (WiC) and word sense disambiguation (WSD). |
Similar Papers
Norm of Word Embedding Encodes Information Gain (2023.emnlp-main)
Copied to clipboard
| Challenge: | Distributed representations of words encode lexical semantic information, but what type of information is encoded and how? |
| Approach: | They propose to use the squared norm of static word embedding to encode the information gain conveyed by the word. |
| Outcome: | The proposed model can encode the information gain of a word in a language model or neural network. |
LTRS: Improving Word Sense Disambiguation via Learning to Rank Senses (2025.coling-main)
Copied to clipboard
| Challenge: | Conventional training strategies only consider predefined senses for target words and learn each of them from relatively limited instances, neglecting the influence of similar ones. |
| Approach: | They propose a method to rank senses to improve the task of word Sense Disambiguation (WSD) by ranking an expanded list of sense definitions. |
| Outcome: | The proposed method achieves a SOTA F1 score of 79.6% in Chinese WSD and shows faster convergence than previous methods. |
Words that make SENSE: Sensorimotor Norms in Learned Lexical Token Representations (2026.findings-acl)
Copied to clipboard
| Challenge: | Empirical studies suggest that comprehending action, perceptual and abstract concepts elicits rapid, automatic activity in modality-specific brain areas. |
| Approach: | They propose a model that predicts Lancaster sensorimotor norms from word lexical embeddings. |
| Outcome: | The proposed model predicts Lancaster sensorimotor norms from word lexical embeddings. |
Sense Embeddings are also Biased – Evaluating Social Biases in Static and Contextualised Sense Embeddings (2022.acl-long)
Copied to clipboard
| Challenge: | Existing studies have evaluated social biases in word embeddings, but they are understudied. |
| Approach: | They propose to evaluate the social biases in sense embeddings using a benchmark dataset for word embedders. |
| Outcome: | The proposed measures show that even when no biases are found at word-level, there are still worrying levels of social biase at sense-level which are often ignored by the word- level bias evaluation measures. |
Language Modelling Makes Sense: Propagating Representations through WordNet for Full-Coverage Word Sense Disambiguation (P19-1)
Copied to clipboard
| Challenge: | Contextual embeddings address the problem of meaning conflation hampering word embeddables. |
| Approach: | They propose a method that creates sense-level embeddings with full-coverage of WordNet without recourse to explicit sense distributions or task-specific modelling. |
| Outcome: | The proposed method surpasses previous systems using powerful models and is robust when ignoring part-of-speech and lemma features. |
Zero-shot Word Sense Disambiguation using Sense Definition Embeddings (P19-1)
Copied to clipboard
| Challenge: | Word Sense Disambiguation (WSD) is an open problem in Natural Language Processing . current methods treat senses as discrete labels and predict the most-frequent-Sense for unseen senses . |
| Approach: | They propose a supervised model to perform Word Sense Disambiguation (WSD) by predicting over a continuous sense embedding space rather than a discrete label space. |
| Outcome: | The proposed model generalizes over seen and unseen senses, achieving zero-shot learning. |
Moving Down the Long Tail of Word Sense Disambiguation with Gloss Informed Bi-encoders (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models for Word Sense Disambiguation are not uniformly distributed on rare or unseen senses. |
| Approach: | They propose a bi-encoder model that embeds the target word with its context and the dictionary definition, or gloss, of each sense. |
| Outcome: | The proposed model outperforms previous state-of-the-art models on English all-words WSD, with a 31.1% error reduction on less frequent senses over prior work. |
Improved Word Sense Disambiguation Using Pre-Trained Contextualized Word Representations (D19-1)
Copied to clipboard
| Challenge: | Contextualized word representations are effective in downstream tasks such as question answering, named entity recognition, and sentiment analysis. |
| Approach: | They propose to integrate pre-trained contextualized word representations into a neural network that captures the whole sentence and the word representation in the sentence. |
| Outcome: | The proposed approach outperforms the state-of-the-art approach that makes use of non-contextualized word embeddings on multiple benchmark WSD datasets. |
Cross-Lingual Word Embedding Refinement by ℓ1 Norm Optimisation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for building high-quality CLWEs learn mappings that minimise the l2 norm loss function but this optimisation objective has been shown to be sensitive to outliers. |
| Approach: | They propose a simple post-processing step to improve cross-lingual word embeddings using the Manhattan norm goodness-of-fit criterion. |
| Outcome: | The proposed approach outperforms four state-of-the-art baselines in bilingual lexicon induction and cross-lingual transfer tasks. |
Semantic Specialization for Knowledge-based Word Sense Disambiguation (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for knowledge-based Word Sense Disambiguation (WSD) use only lexical knowledge to adapt contextualized embeddings. |
| Approach: | They propose a semantic specialization where contextualized embeddings are adapted to the WSD task using only lexical knowledge. |
| Outcome: | The proposed method outperforms previous studies that adapt contextualized embeddings while controlling deviations from the original embeddables. |