Juliano Efson Sales, Leonardo Souza, Siamak Barzegar, Brian Davis, André Freitas, Siegfried Handschuh
| Challenge: | Word embedding/distributional semantic models are a fundamental component in many natural language processing (NLP) architectures. |
| Approach: | They propose a multi-lingual word embedding/distributional semantics framework which supports creation, use and evaluation of word embedded models. |
| Outcome: | The proposed tool supports the creation, use and evaluation of word embedding models. |
Similar Papers
Finely Tuned, 2 Billion Token Based Word Embeddings for Portuguese (L18-1)
Copied to clipboard
| Challenge: | A distributional semantics model is instrumental to improve the performance of many applications and processing tasks for any language. |
| Approach: | They propose to develop an advanced distributional model for Portuguese with the largest vocabulary and best evaluation scores published so far. |
| Outcome: | The proposed model has the largest vocabulary and the best evaluation scores published so far. |
Building Static Embeddings from Contextual Ones: Is It Useful for Building Distributional Thesauri? (2022.lrec-1)
Copied to clipboard
| Challenge: | contextual language models are dominant in the field of Natural Language Processing, but they are not suitable for all uses. |
| Approach: | They propose a method for building word or type-level embeddings from contextual models . they evaluate a large set of English nouns from the perspective of extracting semantic similarity relations . |
| Outcome: | The proposed method can be used to build word or type embeddings from contextual models . it can be exploited for a wide set of English nouns, showing it can improve distributional thesauri . |
A Tour of Explicit Multilingual Semantics: Word Sense Disambiguation, Semantic Role Labeling and Semantic Parsing (2022.aacl-tutorials)
Copied to clipboard
| Challenge: | a recent advent of pretrained language models has sparked a revolution in NLP . but, there are still questions about whether current approaches capture explicit, symbolic meaning . this tutorial will review efforts to tackle three key open problems in lexical and sentence-level semantics . |
| Approach: | This tutorial reviews recent efforts to shed light on meaning in NLP . it will focus on three key open problems in lexical and sentence-level semantics . |
| Outcome: | This tutorial reviews recent efforts to shed light on meaning in NLP . it focuses on three key open problems in lexical and sentence-level semantics . |
On the Correlation of Word Embedding Evaluation Metrics (2020.lrec-1)
Copied to clipboard
| Challenge: | Word embeddings are geometrical representations of word paradigmatics and syntagmatics. |
| Approach: | They propose to investigate evaluation metrics on various datasets to find correlations . they propose a fast solution to select the best word embeddings among many others . |
| Outcome: | The proposed method could be used to select the best word embeddings among many others. |
What’s in Your Embedding, And How It Predicts Task Performance (C18-1)
Copied to clipboard
| Challenge: | Attempts to find a single technique for general-purpose intrinsic evaluation of word embeddings have so far not been successful. |
| Approach: | They propose a method that quantifies interpretable characteristics of word vector neighborhoods and shows how they correlate with performance on 14 extrinsic and intrinsic task datasets. |
| Outcome: | The proposed approach enables multi-faceted evaluation, parameter search, and generally – a more principled, hypothesis-driven approach to development of distributional semantic representations. |
Quantifying Compositionality of Classic and State-of-the-Art Embeddings (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Static word embeddings make strong claims about compositionality, but the SOTA generative models go too far in the other direction. |
| Approach: | a new study evaluates the compositionality of word embeddings by canonical correlation analysis . strong compositional signals are observed in later training stages across data modalities . |
| Outcome: | a new evaluation of compositional models shows that they exploit access meanings when justified . strong compositional signals are observed in later training stages and in deeper layers of the transformer-based model before a decline at the top layer. |
Grounded Compositional Outputs for Adaptive Language Modeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Language models are a key component of natural language processing, but their size is a problem because they are typically trained with a closed output vocabulary derived from the training data. |
| Approach: | They propose a fully compositional output embedding layer for language models that is grounded in semantically related words and free-text definitions. |
| Outcome: | The proposed model outperforms state-of-the-art methods and adaptation approaches on cross-domain modeling and cross-learning tasks. |
Embeddings in Natural Language Processing (2020.coling-tutorials)
Copied to clipboard
| Challenge: | Embeddings have been a key topic of interest in NLP for the past decade . a quick warm-up introduction to NLP and why it is important to have a semantic comprehension of texts . |
| Approach: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and then move to other types of embeddable vectors . |
| Outcome: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and move to other types of embeddable representations . |
RPD: A Distance Function Between Word Embeddings (2020.acl-srw)
Copied to clipboard
| Challenge: | Existing word embeddings are poorly understood, but little is known about how they differ between different sets of word embeds. |
| Approach: | They propose a metric called Relative Pairwise Inner Product Distance to quantify the distance between different word embeddings. |
| Outcome: | The proposed metric measures the distance between different sets of embeddings and investigates the influence of different training processes and corpora. |
Word Embedding Evaluation in Downstream Tasks and Semantic Analogies (2020.lrec-1)
Copied to clipboard
| Challenge: | Language Models (LMs) are an oft studied area of natural language processing . Word Embeddings (WE) are vector space representations of a vocabulary . |
| Approach: | They evaluate Word Embeddings (WE) models for the Portuguese langauage . results show that a diverse corpus can often outperform a larger, less textually diverse corp. |
| Outcome: | The proposed models outperform a larger, less textually diverse corpus in two tasks . the evaluation shows that a diverse and comprehensive corpus outperformed a smaller, less diverse corp. |