CoSimLex: A Resource for Evaluating Graded Word Similarity in Context (2020.lrec-1)
Copied to clipboard
Carlos Santos Armendariz, Matthew Purver, Matej Ulčar, Senja Pollak, Nikola Ljubešić, Mark Granroth-Wilding
| Challenge: | Existing methods to evaluate word embeddings ignore context and treat words in isolation. |
| Approach: | They propose to build a new word embeddings-based dataset that provides context-dependent similarity measures. |
| Outcome: | The proposed dataset provides context-dependent similarity measures and covers a well-resourced language (English) but a number of less-resource languages. |
Similar Papers
On the Correlation of Word Embedding Evaluation Metrics (2020.lrec-1)
Copied to clipboard
| Challenge: | Word embeddings are geometrical representations of word paradigmatics and syntagmatics. |
| Approach: | They propose to investigate evaluation metrics on various datasets to find correlations . they propose a fast solution to select the best word embeddings among many others . |
| Outcome: | The proposed method could be used to select the best word embeddings among many others. |
RAW-C: Relatedness of Ambiguous Words in Context (A New Lexical Resource for English) (2021.acl-long)
Copied to clipboard
| Challenge: | lexical ambiguity is a problem for NLP, but few tasks evaluate its impact on human intuitions. |
| Approach: | They propose to use contextualized word embeddings to evaluate word meaning . they use a dataset of human relatedness judgments and human estimates of sense dominance . |
| Outcome: | The proposed model matches human intuitions with contextualized embeddings on 112 ambiguous words in context with 672 sentence pairs. |
WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations (N19-1)
Copied to clipboard
| Challenge: | Existing word embeddings cannot model the dynamic nature of words’ semantics, i.e., the property of words to correspond to potentially different meanings. |
| Approach: | They propose a large-scale Word in Context dataset, called WiC, which is curated by experts and can be used to evaluate context-sensitive representations. |
| Outcome: | The proposed models outperform the standard evaluation dataset for the purpose and highlight their shortcomings. |
Improved Word Sense Disambiguation Using Pre-Trained Contextualized Word Representations (D19-1)
Copied to clipboard
| Challenge: | Contextualized word representations are effective in downstream tasks such as question answering, named entity recognition, and sentiment analysis. |
| Approach: | They propose to integrate pre-trained contextualized word representations into a neural network that captures the whole sentence and the word representation in the sentence. |
| Outcome: | The proposed approach outperforms the state-of-the-art approach that makes use of non-contextualized word embeddings on multiple benchmark WSD datasets. |
A Deeper Look into Dependency-Based Word Embeddings (N18-4)
Copied to clipboard
| Challenge: | Word embeddings trained with dependency contexts excel at different tasks, and enhanced dependencies often improve performance. |
| Approach: | They propose to use dependency-based word embeddings to capture semantic similarity rather than relatedness. |
| Outcome: | The results show that word embeddings trained with Universal and Stanford dependencies excel at different tasks and that enhanced dependencies often improve performance. |
Word Sense Disambiguation: Towards Interactive Context Exploitation from Both Word and Sense Perspectives (2021.acl-long)
Copied to clipboard
| Challenge: | Recent Word Sense Disambiguation systems have approached the upper bound of the task on standard evaluation benchmarks. |
| Approach: | They propose to convert the nearly isolated decisions into interrelated ones by exposing senses in context when learning sense embeddings in a similarity-based Sense Aware Context Exploitation architecture. |
| Outcome: | The proposed approach surpasses state-of-the-art on English and multilingual datasets by large margins. |
Exploring the Representation of Word Meanings in Context: A Case Study on Homonymy and Synonymy (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models that represent different senses of words in context are not accurate for polysemous words. |
| Approach: | They propose a multilingual dataset that evaluates the ability of models to accurately represent different lexical-semantic relations such as homonymy and synonymy. |
| Outcome: | The proposed models can disambiguate homonyms in context, but fail to represent words with different senses when occurring in similar sentences. |
A Multilingual Evaluation Dataset for Monolingual Word Sense Alignment (2020.lrec-1)
Copied to clipboard
Sina Ahmadi, John Philip McCrae, Sanni Nimb, Fahad Khan, Monica Monachini, Bolette Pedersen, Thierry Declerck, Tanja Wissik, Andrea Bellandi, Irene Pisani, Thomas Troelsgård, Sussi Olsen, Simon Krek, Veronika Lipp, Tamás Váradi, László Simon, András Gyorffy, Carole Tiberius, Tanneke Schoonheim, Yifat Ben Moshe, Maya Rudich, Raya Abu Ahmad, Dorielle Lonke, Kira Kovalenko, Margit Langemets, Jelena Kallas, Oksana Dereza, Theodorus Fransen, David Cillessen, David Lindemann, Mikel Alonso, Ana Salgado, José Luis Sancho, Rafael-J. Ureña-Ruiz, Jordi Porta Zamorano, Kiril Simov, Petya Osenova, Zara Kancheva, Ivaylo Radev, Ranka Stanković, Andrej Perdih, Dejan Gabrovsek
| Challenge: | a new dataset aims to align monolingual dictionaries with a single sense level for 15 languages . this dataset covers a wide range of languages and resources . |
| Approach: | They propose to manually align monolingual dictionaries with possible semantic relationships . they use 15 languages to create a new baseline for the task of monolingual word sense alignment . |
| Outcome: | The proposed dataset covers 15 languages and covers the more challenging task of linking general-purpose language. |
Patterns of Polysemy and Homonymy in Contextualised Language Models (2021.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study has focused on homonymy, a variety of multiplicity of meanings exemplified by word forms with unrelated meanings. |
| Approach: | They investigate the extent to which contextualised embeddings reflect traditional distinctions of polysemy and homonymy. |
| Outcome: | The proposed model shows that it can distinguish between polysemy and homonymy . it shows that the model fails to replicate the results of the human-annotated dataset . |
Rethinking Word Similarity: Semantic Similarity through Classification Confusion (2025.naacl-long)
Copied to clipboard
| Challenge: | Word similarity measures cannot capture context-dependent, asymmetrical, polysemous nature of semantic similarity. |
| Approach: | They propose a new measure of similarity that reframes semantic similarity in terms of feature-based classification confusion. |
| Outcome: | The proposed model is comparable to cosine similarity in matching human similarity judgments across several datasets and can measure similarity using predetermined features of interest. |