AMenDeD: Modelling Concepts by Aligning Mentions, Definitions and Decontextualised Embeddings (2024.lrec-main)
Copied to clipboard
| Challenge: | Contextualised Language Models (LMs) improve on word embeddings by encoding meaning of words in context. |
| Approach: | They propose to learn a unified embedding space in which all three types of representations can be integrated. |
| Outcome: | The proposed model outperforms existing approaches in ontology completion tasks. |
Similar Papers
Towards Better Context-aware Lexical Semantics:Adjusting Contextualized Representations through Static Anchors (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has shown that contextualized models generate dynamic embeddings for words in context, but static embedds are often overlooked in this trend towards contextualized modeling. |
| Approach: | They propose a method that learns a transformation through static anchors and requires only another pre-trained model. |
| Outcome: | The proposed method improves a range of benchmark tasks that test contextual variations of meaning across different usages of a word and across different words as they are used in context. |
Building Static Embeddings from Contextual Ones: Is It Useful for Building Distributional Thesauri? (2022.lrec-1)
Copied to clipboard
| Challenge: | contextual language models are dominant in the field of Natural Language Processing, but they are not suitable for all uses. |
| Approach: | They propose a method for building word or type-level embeddings from contextual models . they evaluate a large set of English nouns from the perspective of extracting semantic similarity relations . |
| Outcome: | The proposed method can be used to build word or type embeddings from contextual models . it can be exploited for a wide set of English nouns, showing it can improve distributional thesauri . |
Language Modelling Makes Sense: Propagating Representations through WordNet for Full-Coverage Word Sense Disambiguation (P19-1)
Copied to clipboard
| Challenge: | Contextual embeddings address the problem of meaning conflation hampering word embeddables. |
| Approach: | They propose a method that creates sense-level embeddings with full-coverage of WordNet without recourse to explicit sense distributions or task-specific modelling. |
| Outcome: | The proposed method surpasses previous systems using powerful models and is robust when ignoring part-of-speech and lemma features. |
Negation, Coordination, and Quantifiers in Contextualized Language Models (2022.coling-1)
Copied to clipboard
| Challenge: | Recent work has focused on specific tasks and on the learning outcome. |
| Approach: | They propose to decouple the weaknesses from specific tasks and focus on the embeddings per se and their mode of learning. |
| Outcome: | The proposed model can learn semantic constraints and how the context impacts their embeddings. |
Partial Colexifications Improve Concept Embeddings (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for embedding words from colexification networks are limited to the word level, ignoring lexical relations that would only hold for parts of words in a given language. |
| Approach: | They propose to embed concepts from automatically constructed colexification networks . they use lexical similarity ratings and word association data to evaluate the methods . |
| Outcome: | The proposed methods capture and represent different semantic relationships between concepts. |
Deep Generative Model for Joint Alignment and Word Representation (N18-1)
Copied to clipboard
| Challenge: | EmbedAlign model embeds words in their complete observed context and learns by marginalisation of latent lexical alignments. |
| Approach: | They exploit translation as a distributional context and embed words as posterior probability densities, rather than point estimates, which allows them to compare words in context using a measure of overlap between distributions. |
| Outcome: | The proposed model performs on a range of lexical semantics tasks and achieves competitive results on benchmarks including natural language inference, paraphrasing, and text similarity. |
CALE : Concept-Aligned Embeddings for Both Within-Lemma and Inter-Lemma Sense Differentiation (2026.eacl-long)
Copied to clipboard
| Challenge: | Recent work on Word-in-Context fine-tunes models to investigate lexical meaning but only compares occurrences of the same lemma, limiting the range of captured information. |
| Approach: | They propose an extension to Word-in-Context to include inter-words scenarios by using a dataset and several models on a data set. |
| Outcome: | The proposed models provide efficient multi-purpose representations of lexical meaning that reach best performances in the experiments. |
VCDM: Leveraging Variational Bi-encoding and Deep Contextualized Word Representations for Improved Definition Modeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches for definition modeling combine distributional and lexical semantics in an implicit rather than direct way. |
| Approach: | They propose a model that introduces a continuous latent variable to model the relationship between a phrase and its definition. |
| Outcome: | The proposed model achieves state-of-the-art performance on four challenging benchmarks and the first non-English corpus. |
Word Alignment by Fine-tuning Embeddings on Parallel Corpora (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing work on word alignment has focused on unsupervised learning on parallel text. |
| Approach: | They propose to combine pre-trained contextualized word embeddings with multilingually trained language models to achieve competitive results on word alignment tasks. |
| Outcome: | The proposed model outperforms state-of-the-art models on five language pairs and can train multilingual word aligners that can obtain robust performance on different language pairs. |
Learning High-Quality and General-Purpose Phrase Representations (2024.findings-eacl)
Copied to clipboard
| Challenge: | Pre-trained language models for phrasal embeddings are unnecessarily complex and require to be pre-tuned on a corpus with context sentences. |
| Approach: | They propose a framework to learn phrase representations in a context-free fashion. |
| Outcome: | The proposed framework generates superior phrase embeddings while requiring a smaller model size. |