SensePOLAR: Word sense aware interpretability for pre-trained contextual word embeddings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing word embedding models lack interpretability for words . |
| Approach: | They propose to add interpretability to word embeddings by using a POLAR framework that enables wordsense aware interpretations for pre-trained contextual word embeds. |
| Outcome: | The proposed framework achieves comparable performance to existing embeddings across GLUE and SQuAD benchmarks. |
Similar Papers
Word2Sense: Sparse Interpretable Word Embeddings (P19-1)
Copied to clipboard
| Challenge: | Word2Sense embeddings are interpretable, but they are sparse and fast to compute . a unitary rotation can be applied to many of these embeddables retaining their utility for computational tasks while changing the values of individual coordinates. |
| Approach: | They propose an unsupervised method to generate Word2Sense word embeddings that are interpretable. |
| Outcome: | The proposed method compares well with other unsupervised word embeddings on NLP tasks. |
With More Contexts Comes Better Performance: Contextualized Sense Embeddings for All-Round Word Sense Disambiguation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Contextualized word embeddings have been used effectively across several tasks in Natural Language Processing, but it is difficult to link them to structured sources of knowledge. |
| Approach: | They propose a semi-supervised approach to producing sense embeddings for the lexical meanings within a lexicon that is comparable to that of contextualized word vectors. |
| Outcome: | The proposed approach outperforms state-of-the-art models in the English Word Sense Disambiguation task and in the multilingual one while training on sense-annotated data in English only. |
Improved Word Sense Disambiguation Using Pre-Trained Contextualized Word Representations (D19-1)
Copied to clipboard
| Challenge: | Contextualized word representations are effective in downstream tasks such as question answering, named entity recognition, and sentiment analysis. |
| Approach: | They propose to integrate pre-trained contextualized word representations into a neural network that captures the whole sentence and the word representation in the sentence. |
| Outcome: | The proposed approach outperforms the state-of-the-art approach that makes use of non-contextualized word embeddings on multiple benchmark WSD datasets. |
Which Evaluations Uncover Sense Representations that Actually Make Sense? (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing sense representations fail for human-centric tasks like inspecting a language’s sense inventory. |
| Approach: | They propose a coherence evaluation for sense embeddings and a model optimized for finding interpretable sense representations that are more coherent than existing sense embeds. |
| Outcome: | The proposed model is more coherent than existing sense embeddings and offers comparable word similarities with multisense representations while learning more distinguishable, interpretable senses. |
Advances in Pre-Training Distributed Word Representations (L18-1)
Copied to clipboard
| Challenge: | Pre-trained word representations are a building block of many Natural Language Processing and Machine Learning applications. |
| Approach: | They propose to combine known tricks and a set of publicly available pre-trained word vector representations to train high-quality representations. |
| Outcome: | The proposed models outperform the current state of the art on a number of tasks while maintaining a high training speed to scale to massive amount of data. |
Are Embedding Spaces Interpretable? Results of an Intrusion Detection Evaluation on a Large French Corpus (2022.lrec-1)
Copied to clipboard
| Challenge: | Word embedding methods use word co-occurrences to encode, syntactic and semantic information to describe vocabulary in a low-dimensional space. |
| Approach: | They evaluate word embedding interpretability using two methods . they use a word-in-space vector encoder and graph-based method SPINE . |
| Outcome: | The proposed methods show that they can be interpretable on a large French corpus. |
Sense Embeddings are also Biased – Evaluating Social Biases in Static and Contextualised Sense Embeddings (2022.acl-long)
Copied to clipboard
| Challenge: | Existing studies have evaluated social biases in word embeddings, but they are understudied. |
| Approach: | They propose to evaluate the social biases in sense embeddings using a benchmark dataset for word embedders. |
| Outcome: | The proposed measures show that even when no biases are found at word-level, there are still worrying levels of social biase at sense-level which are often ignored by the word- level bias evaluation measures. |
Joint Learning of Sense and Word Embeddings (L18-1)
Copied to clipboard
| Challenge: | Existing methods for learning lower-dimensional representations of words using unlabelled data learn a single representation for a word, ignoring the different senses of that word (polysemy). |
| Approach: | They propose a method that jointly learns sense-aware word embeddings using both unlabelled and sense-tagged text corpora. |
| Outcome: | The proposed method outperforms competing methods on word similarity and short-text classification benchmark datasets. |
Language Modelling Makes Sense: Propagating Representations through WordNet for Full-Coverage Word Sense Disambiguation (P19-1)
Copied to clipboard
| Challenge: | Contextual embeddings address the problem of meaning conflation hampering word embeddables. |
| Approach: | They propose a method that creates sense-level embeddings with full-coverage of WordNet without recourse to explicit sense distributions or task-specific modelling. |
| Outcome: | The proposed method surpasses previous systems using powerful models and is robust when ignoring part-of-speech and lemma features. |
Embeddings in Natural Language Processing (2020.coling-tutorials)
Copied to clipboard
| Challenge: | Embeddings have been a key topic of interest in NLP for the past decade . a quick warm-up introduction to NLP and why it is important to have a semantic comprehension of texts . |
| Approach: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and then move to other types of embeddable vectors . |
| Outcome: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and move to other types of embeddable representations . |