XL-LEXEME: WiC Pretrained Model for Cross-Lingual LEXical sEMantic changE (2023.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to the Word in Context task use cross-encoders, which prevent the possibility of deriving comparable word embeddings. |
| Approach: | They propose a Lexical Semantic Change Detection model that extends SBERT, highlighting the target word in the sentence. |
| Outcome: | The proposed model outperforms the state-of-the-art on the multilingual benchmarks for SemEval-2020 Task 1 - Lexical Semantic Change (LSC) Detection and the RuShiftEval shared task. |
Similar Papers
A Systematic Comparison of Contextualized Word Embeddings for Lexical Semantic Change (2024.naacl-long)
Copied to clipboard
| Challenge: | Contextualized embeddings are the preferred tool for modeling Lexical Semantic Change (LSC) current evaluations focus on a specific task known as Graded Change Detection (GCD) however, performance comparisons between different approaches are often misleading due to diverse settings. |
| Approach: | They evaluate the performance of contextualized embeddings for Lexical Semantic Change (LSC) they break the problem into Word-in-Context (WiC) and Word Sense Induction (WSI) tasks . |
| Outcome: | The proposed model outperforms other models on eight available benchmarks for Lexical Semantic Change (LSC) while comparable to GPT-4. |
Effects of Pre- and Post-Processing on type-based Embeddings in Lexical Semantic Change Detection (2021.eacl-main)
Copied to clipboard
| Challenge: | Lexical semantic change detection is a new and innovative research field. |
| Approach: | They propose to pre-train on large corpora and refine on diachronic target corpors to improve performance. |
| Outcome: | The proposed models improve on large corpora and diachronic target corpors . the proposed models are compared with existing models in a variety of learning scenarios . |
XL-WiC: A Multilingual Benchmark for Evaluating Semantic Contextualization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation benchmarks for assessing distinct meanings of words are tied to sense inventories, restricting their usage to knowledge-based representation techniques. |
| Approach: | They propose a multilingual benchmark that models distinct meanings of words in English . they use a binary disambiguation task with gold standards in 12 new languages . |
| Outcome: | The proposed model can model distinct meanings of words in English even when no tagged instances are available for a target language. |
Explaining and Improving BERT Performance on Lexical Semantic Change Detection (2021.eacl-srw)
Copied to clipboard
| Challenge: | Lexical semantic change detection is still a challenging field due to the success of type-based embeddings in SemEval-2020 Task 1 and other NLP tasks. |
| Approach: | They compare the performance of BERT embeddings with results from the word sense disambiguation dataset underlying SemEval-2020 Task 1 and the Italian follow-up task DIACR-Ita. |
| Outcome: | The proposed model outperforms token-based embeddings on lexical semantic change detection tasks. |
A Semantic Distance Metric Learning approach for Lexical Semantic Change Detection (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing Word-in-Context (WiC) datasets are used to detect temporal semantic changes of words. |
| Approach: | They propose a supervised two-staged SCD method that uses existing Word-in-Context (WiC) datasets to predict temporal semantic changes of words. |
| Outcome: | The proposed method achieves strong performance in multiple languages and significant improvements on WiC benchmarks. |
Analysing Lexical Semantic Change with Contextualised Word Representations (2020.acl-main)
Copied to clipboard
| Challenge: | Existing studies on lexical semantic change have focused on detecting and characterising word meaning shifts using distributional semantic models. |
| Approach: | They propose a method that exploits the BERT neural language model to obtain representations of word usages, clusters these representations into usage types, and measures change along time with three proposed metrics. |
| Outcome: | The proposed method captures a variety of synchronic and diachronic linguistic phenomena and is highly reproducible and reproducible. |
Definition generation for lexical semantic change detection (2024.findings-acl)
Copied to clipboard
| Challenge: | a number of studies have attempted to bridge the gap between lexical semantic change detection and sense-based LSCD methods. |
| Approach: | They propose a sense distribution based LSCD method which uses contextualized word definitions as 'senses' they argue that the method preserves interpretability and allows to inspect the reasons behind a specific shift in terms of discrete definitions-as-sense. |
| Outcome: | The proposed method outperforms previous sense-based methods on five datasets and three languages and preserves interpretability and allows to inspect the reasons behind a specific shift in terms of discrete definitions-as-senses. |
LexFit: Lexical Fine-Tuning of Pretrained Language Models (2021.acl-long)
Copied to clipboard
| Challenge: | Transformer-based language models implicitly store a wealth of lexical semantic knowledge, but it is non-trivial to extract that knowledge effectively from their parameters. |
| Approach: | They propose to expose and enrich lexical knowledge from transformer-based language models to serve as effective decontextualized word encoders even when fed input words "in isolation" |
| Outcome: | The proposed model outperforms standard static WEs and vanilla LMs in lexical tasks over four established tasks in 8 languages. |
LSC-Eval: A General Framework to Evaluate Methods for Assessing Dimensions of Lexical Semantic Change Using LLM-Generated Synthetic Data (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for measuring Lexical Semantic Change are lacking historical benchmarks. |
| Approach: | They propose a three-stage general-purpose evaluation framework that simulates theory-driven LSC using In-Context Learning and a lexical database. |
| Outcome: | The proposed framework evaluates the sensitivity of computational methods to synthetic change and their suitability for detecting change in specific dimensions and domains. |
Injecting Wiktionary to improve token-level contextual representations using contrastive learning (2024.eacl-short)
Copied to clipboard
| Challenge: | lexical semantics tasks require contextual word embeddings that are not blind to context, despite the fact that vectors of the same meaning are too different. |
| Approach: | They propose to fine-tune pre-trained language models by using automatically self-augmented examples to target contextual word embeddings. |
| Outcome: | The proposed method achieves significant improvements on the original WiC test set and in two new tests. |