WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations (N19-1)
Copied to clipboard
| Challenge: | Existing word embeddings cannot model the dynamic nature of words’ semantics, i.e., the property of words to correspond to potentially different meanings. |
| Approach: | They propose a large-scale Word in Context dataset, called WiC, which is curated by experts and can be used to evaluate context-sensitive representations. |
| Outcome: | The proposed models outperform the standard evaluation dataset for the purpose and highlight their shortcomings. |
Similar Papers
XL-WiC: A Multilingual Benchmark for Evaluating Semantic Contextualization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation benchmarks for assessing distinct meanings of words are tied to sense inventories, restricting their usage to knowledge-based representation techniques. |
| Approach: | They propose a multilingual benchmark that models distinct meanings of words in English . they use a binary disambiguation task with gold standards in 12 new languages . |
| Outcome: | The proposed model can model distinct meanings of words in English even when no tagged instances are available for a target language. |
WiC-TSV: An Evaluation Benchmark for Target Sense Verification of Words in Context (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing benchmarks for Word Sense Disambiguation are limited to those systems in which sense distinctions are defined according to an underlying sense inventory. |
| Approach: | They propose a framework for Target Sense Verification of Words in Context which grounds its uniqueness as binary classification task and independent of external sense inventories. |
| Outcome: | The proposed framework is highly flexible for evaluation of diverse models and systems in and across domains. |
Improved Word Sense Disambiguation Using Pre-Trained Contextualized Word Representations (D19-1)
Copied to clipboard
| Challenge: | Contextualized word representations are effective in downstream tasks such as question answering, named entity recognition, and sentiment analysis. |
| Approach: | They propose to integrate pre-trained contextualized word representations into a neural network that captures the whole sentence and the word representation in the sentence. |
| Outcome: | The proposed approach outperforms the state-of-the-art approach that makes use of non-contextualized word embeddings on multiple benchmark WSD datasets. |
Injecting Wiktionary to improve token-level contextual representations using contrastive learning (2024.eacl-short)
Copied to clipboard
| Challenge: | lexical semantics tasks require contextual word embeddings that are not blind to context, despite the fact that vectors of the same meaning are too different. |
| Approach: | They propose to fine-tune pre-trained language models by using automatically self-augmented examples to target contextual word embeddings. |
| Outcome: | The proposed method achieves significant improvements on the original WiC test set and in two new tests. |
With More Contexts Comes Better Performance: Contextualized Sense Embeddings for All-Round Word Sense Disambiguation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Contextualized word embeddings have been used effectively across several tasks in Natural Language Processing, but it is difficult to link them to structured sources of knowledge. |
| Approach: | They propose a semi-supervised approach to producing sense embeddings for the lexical meanings within a lexicon that is comparable to that of contextualized word vectors. |
| Outcome: | The proposed approach outperforms state-of-the-art models in the English Word Sense Disambiguation task and in the multilingual one while training on sense-annotated data in English only. |
Dynamic Contextualized Word Embeddings (2021.acl-long)
Copied to clipboard
| Challenge: | Static word embeddings that represent words by a single vector cannot capture word meaning in different linguistic and extralinguistic contexts. |
| Approach: | They propose dynamic contextualized word embeddings that represent words as a function of linguistic and extralinguistic contexts. |
| Outcome: | The proposed model models time and social space jointly, making them attractive for NLP tasks involving semantic variability. |
Exploring the Value of Personalized Word Embeddings (2020.coling-main)
Copied to clipboard
| Challenge: | a subset of words belonging to specific psycholinguistic categories vary more in their representations across users . combining generic and personalized word embeddings yields the best performance . |
| Approach: | They propose personalized word embeddings and compare their performance to generic ones . they show that personalized word representations can be leveraged for improved performance . |
| Outcome: | The proposed model can be used for authorship attribution. |
Enhancing the Context Representation in Similarity-based Word Sense Disambiguation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing similarity-based systems focus on learning sense embeddings using only the sentence where the word appears, neglecting its global context. |
| Approach: | They propose a contextoriented embedding technique that takes better advantage of both word-level and sense-level global context of an ambiguous word for disambiguation. |
| Outcome: | The proposed method improves on all-words WSD benchmarks in knowledge-based category by large margins. |
A Systematic Comparison of Contextualized Word Embeddings for Lexical Semantic Change (2024.naacl-long)
Copied to clipboard
| Challenge: | Contextualized embeddings are the preferred tool for modeling Lexical Semantic Change (LSC) current evaluations focus on a specific task known as Graded Change Detection (GCD) however, performance comparisons between different approaches are often misleading due to diverse settings. |
| Approach: | They evaluate the performance of contextualized embeddings for Lexical Semantic Change (LSC) they break the problem into Word-in-Context (WiC) and Word Sense Induction (WSI) tasks . |
| Outcome: | The proposed model outperforms other models on eight available benchmarks for Lexical Semantic Change (LSC) while comparable to GPT-4. |
Reassessing Semantic Knowledge Encoded in Large Language Models through the Word-in-Context Task (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have propelled significant progress, extending their application across various domains including dialogue systems, text generation, translation systems, and beyond. |
| Approach: | They propose to use the Word-in-Context (WiC) task to reassess the semantic knowledge encoded in large language models (LLMs) they prompt LLMs to generate natural language descriptions that contrast the meanings of the target word in two contextual sentences given in the WiC dataset. |
| Outcome: | The proposed model significantly improves the classification accuracy of the two models. |