To Word Senses and Beyond: Inducing Concepts with Contextualized Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Word Sense Disambiguiation and Word sense Induction are considered independent problems, but they are often neglected in practice. |
| Approach: | They propose an unsupervised task of learning a soft clustering amongwords that defines a set of concepts directly from data. |
| Outcome: | The proposed approach leverages both a local and global cross-lexicon view to induce concepts and also senses in the context of the proposed task. |
Similar Papers
PolyLM: Learning about Polysemy through Language Modeling (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods to embed word senses have been overtaken by contextualized embeddings . alan ansell and jim koenig present a method which can be applied to downstream tasks . |
| Approach: | They propose a method which formulates learning sense embeddings as a language modeling problem. |
| Outcome: | The proposed method performs better than existing sense embedding methods on WSI tasks . it matches the current state-of-the-art specialized WSi method despite having six times fewer parameters . |
Large Scale Substitution-based Word Sense Induction (2022.acl-long)
Copied to clipboard
| Challenge: | Word forms are ambiguous, and derive meaning from the context in which they appear . word sense induction can be performed over a corpus-derived sense inventory . |
| Approach: | They propose a word-sense induction method based on pre-trained masked language models . they train a static word embeddings algorithm on the sense-tagged corpus . |
| Outcome: | The proposed method outperforms existing senseful embeddings methods on Wikipedia and on an outlier detection dataset. |
In the LLM era, Word Sense Induction remains unsolved (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing word sense induction methods rely on predefined sense inventories of arbitrary granularity and require labor intensive sense-annotated data. |
| Approach: | They propose to evaluate an LLM-based word sense induction method for English . they use data augmentation sources, corpus and lexicon, and semi-supervised scenarios to augment the data. |
| Outcome: | The proposed method surpasses the previous SOTA system on a English set by 3.3%. |
Word sense extension (2023.acl-long)
Copied to clipboard
| Challenge: | a long-standing effort in natural language processing has focused on word sense disambiguation, but little has been explored about how word meaning is extended toward new context. |
| Approach: | They propose a framework that partitions a word type into two pseudo-tokens that mark its different senses and infers whether the meaning can be extended to convey the sense denoted by the token. |
| Outcome: | The proposed framework outperforms other models in predicting plausible novel senses for over 7,500 English words. |
Leveraging Gloss Knowledge in Neural Word Sense Disambiguation by Hierarchical Co-Attention (D18-1)
Copied to clipboard
| Challenge: | Existing models for Word Sense Disambiguation use labeled data, but lack gloss knowledge. |
| Approach: | They propose a co-attention mechanism to generate co-dependent representations for context and gloss . they propose to incorporate gloss knowledge into neural networks for Word Sense Disambiguation . |
| Outcome: | The proposed model achieves state-of-the-art results on standard English all-words WSD datasets. |
Language Modelling Makes Sense: Propagating Representations through WordNet for Full-Coverage Word Sense Disambiguation (P19-1)
Copied to clipboard
| Challenge: | Contextual embeddings address the problem of meaning conflation hampering word embeddables. |
| Approach: | They propose a method that creates sense-level embeddings with full-coverage of WordNet without recourse to explicit sense distributions or task-specific modelling. |
| Outcome: | The proposed method surpasses previous systems using powerful models and is robust when ignoring part-of-speech and lemma features. |
Latent semantic network induction in the context of linked example senses (D19-55)
Copied to clipboard
| Challenge: | Using the Princeton WordNet, we construct a network using the entirety of Wiktionary. |
| Approach: | They propose to use Wiktionary to construct a wordnet using the entirety of the open-source dictionary. |
| Outcome: | The proposed network induction process is similar to the Princeton WordNet, but with a more data-driven approach. |
Improved Word Sense Disambiguation Using Pre-Trained Contextualized Word Representations (D19-1)
Copied to clipboard
| Challenge: | Contextualized word representations are effective in downstream tasks such as question answering, named entity recognition, and sentiment analysis. |
| Approach: | They propose to integrate pre-trained contextualized word representations into a neural network that captures the whole sentence and the word representation in the sentence. |
| Outcome: | The proposed approach outperforms the state-of-the-art approach that makes use of non-contextualized word embeddings on multiple benchmark WSD datasets. |
A Neural Generative Model for Joint Learning Topics and Topic-Specific Word Embeddings (2020.tacl-1)
Copied to clipboard
| Challenge: | Experimental results show that the proposed model outperforms word-level embedding methods in word similarity evaluation and word sense disambiguation. |
| Approach: | They propose a generative model that explores local and global context for joint learning topics and topic-specific word embeddings. |
| Outcome: | The proposed model outperforms word-level embedding methods in word similarity evaluation and word sense disambiguation. |
SensePOLAR: Word sense aware interpretability for pre-trained contextual word embeddings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing word embedding models lack interpretability for words . |
| Approach: | They propose to add interpretability to word embeddings by using a POLAR framework that enables wordsense aware interpretations for pre-trained contextual word embeds. |
| Outcome: | The proposed framework achieves comparable performance to existing embeddings across GLUE and SQuAD benchmarks. |