Diachronic Sense Modeling with Deep Contextualized Word Embeddings: An Ecological View (P19-1)
Copied to clipboard
| Challenge: | Existing word embeddings only assign one vector to a word for a time period, thus they face the meaning conflation deficiency. |
| Approach: | They propose a sense representation and tracking framework based on deep contextualized embeddings that can be used to answer what and when the word meaning changes. |
| Outcome: | The proposed framework is effective in representing fine-grained word senses, and brings a significant improvement in word change detection task. |
Similar Papers
Diachronic word embeddings and semantic shifts: a survey (C18-1)
Copied to clipboard
| Challenge: | Existing methods for tracing time-related semantic shifts with word embedding models lack the cohesion, common terminology and shared practices of more established areas of natural language processing. |
| Approach: | They propose several axes along which these methods can be compared and propose a framework for comparison. |
| Outcome: | The proposed methods are compared with existing methods and outline their main challenges and potential applications. |
Leveraging Contextual Embeddings for Detecting Diachronic Semantic Shift (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for word embeddings have been used to model semantic relations with word embeds. |
| Approach: | They propose a method that leverages contextual embeddings for diachronic semantic shift detection by generating time specific word representations from BERT embedds. |
| Outcome: | The proposed method performs comparable to the current state-of-the-art without time consuming domain adaptation on large corpora. |
Neural Temporality Adaptation for Document Classification: Diachronic Word Embeddings and Domain Adaptation Models (P19-1)
Copied to clipboard
| Challenge: | Recent studies show that document classifiers can become more stable over time when trained in ways that account for temporal variations. |
| Approach: | They propose a method for embedding diachronic word embedds into document classification models . they propose 'time-driven neural classification model' that accounts for temporal variations . |
| Outcome: | The proposed model can be trained on six corpora and make it more robust over time. |
Sequential Modelling of the Evolution of Word Representations for Semantic Change Detection (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that detect semantically shifted words do not account for its evolution through time. |
| Approach: | They propose three variants of sequential models for detecting semantically shifted words . they demonstrate that temporal modelling of word representations yields a clear-cut advantage . |
| Outcome: | The proposed models account for the changes in word representations over time. |
Exploring Word Usage Change with Continuously Evolving Embeddings (2021.acl-demo)
Copied to clipboard
| Challenge: | a new method to track word usage changes is proposed for text datasets that are collected over a longer period of time. |
| Approach: | They propose a way to track word usage changes via continuously evolving embeddings . they demonstrate an interactive web app that can explore semantic shifts with interactive plots a text . |
| Outcome: | The proposed method can be used to analyze word usage changes with interactive plots. |
Dynamic Contextualized Word Embeddings (2021.acl-long)
Copied to clipboard
| Challenge: | Static word embeddings that represent words by a single vector cannot capture word meaning in different linguistic and extralinguistic contexts. |
| Approach: | They propose dynamic contextualized word embeddings that represent words as a function of linguistic and extralinguistic contexts. |
| Outcome: | The proposed model models time and social space jointly, making them attractive for NLP tasks involving semantic variability. |
Analyzing Continuous Semantic Shifts with Diachronic Word Similarity Matrices (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods to analyze word sense proportions are insufficient for understanding semantic shifts . et al., 2018: semantic shift and its effects. |
| Approach: | They propose a framework for how semantic shifts occur over multiple time periods by using word embeddings. |
| Outcome: | The proposed framework can analyze semantic shifts over multiple time periods using word embeddings. |
Improved Word Sense Disambiguation Using Pre-Trained Contextualized Word Representations (D19-1)
Copied to clipboard
| Challenge: | Contextualized word representations are effective in downstream tasks such as question answering, named entity recognition, and sentiment analysis. |
| Approach: | They propose to integrate pre-trained contextualized word representations into a neural network that captures the whole sentence and the word representation in the sentence. |
| Outcome: | The proposed approach outperforms the state-of-the-art approach that makes use of non-contextualized word embeddings on multiple benchmark WSD datasets. |
SensePOLAR: Word sense aware interpretability for pre-trained contextual word embeddings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing word embedding models lack interpretability for words . |
| Approach: | They propose to add interpretability to word embeddings by using a POLAR framework that enables wordsense aware interpretations for pre-trained contextual word embeds. |
| Outcome: | The proposed framework achieves comparable performance to existing embeddings across GLUE and SQuAD benchmarks. |
Deep Generative Model for Joint Alignment and Word Representation (N18-1)
Copied to clipboard
| Challenge: | EmbedAlign model embeds words in their complete observed context and learns by marginalisation of latent lexical alignments. |
| Approach: | They exploit translation as a distributional context and embed words as posterior probability densities, rather than point estimates, which allows them to compare words in context using a measure of overlap between distributions. |
| Outcome: | The proposed model performs on a range of lexical semantics tasks and achieves competitive results on benchmarks including natural language inference, paraphrasing, and text similarity. |