| Challenge: | Existing research on justifying additive compositionality of word embedding models requires a rather strong assumption of uniform word distribution. |
| Approach: | They propose to relax the assumption of uniform word distribution and propose more realistic conditions for proving additive compositionality. |
| Outcome: | The proposed model improves on word similarity and noisy sentence similarity. |
Similar Papers
Quantifying Compositionality of Classic and State-of-the-Art Embeddings (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Static word embeddings make strong claims about compositionality, but the SOTA generative models go too far in the other direction. |
| Approach: | a new study evaluates the compositionality of word embeddings by canonical correlation analysis . strong compositional signals are observed in later training stages across data modalities . |
| Outcome: | a new evaluation of compositional models shows that they exploit access meanings when justified . strong compositional signals are observed in later training stages and in deeper layers of the transformer-based model before a decline at the top layer. |
How do Transformer Embeddings Represent Compositions? A Functional Analysis (2025.findings-acl)
Copied to clipboard
| Challenge: | Despite the popularity of transformer-based models, little is known about how they represent compound words and whether they are compositional. |
| Approach: | They evaluate compositionality in mistral, OpenAI Large, and Google embedding models and compare them with BERT. |
| Outcome: | The proposed models perform best in addition, multiplication, dilation, regression, and the classic vector addition model performs almost as well as any other model. |
Enhancing Word Embeddings with Knowledge Extracted from Lexical Resources (2020.acl-srw)
Copied to clipboard
| Challenge: | In this paper, we present an effective method for semantic specialization of word vector representations. |
| Approach: | They propose a method for semantic specialization of word vector representations using BabelNet. |
| Outcome: | The proposed method improves on word similarity and dialog state tracking tasks. |
Exploring the Value of Personalized Word Embeddings (2020.coling-main)
Copied to clipboard
| Challenge: | a subset of words belonging to specific psycholinguistic categories vary more in their representations across users . combining generic and personalized word embeddings yields the best performance . |
| Approach: | They propose personalized word embeddings and compare their performance to generic ones . they show that personalized word representations can be leveraged for improved performance . |
| Outcome: | The proposed model can be used for authorship attribution. |
On the Compositionality Prediction of Noun Phrases using Poincaré Embeddings (P19-1)
Copied to clipboard
| Challenge: | idiomatic phrases have a non-compositional meaning, meanings of which can be derived from constituents and their grammatical relations. |
| Approach: | They propose to combine hierarchical and distributional information to blend hierarchic and distribution-based hierarchies to detect compositionality for noun phrases. |
| Outcome: | The proposed technique achieves significant improvements over state-of-the-art models based on distributional information and a weighted average of the distributional similarity and p-like function. |
Simple and effective data augmentation for compositional generalization (2024.naacl-long)
Copied to clipboard
| Challenge: | Compositional generalization is the ability of a system to correctly predict the meaning of complex sentences when trained on simpler sentences. |
| Approach: | They propose to use data augmentation methods to generate additional training data by sampling from an augmentation distribution to generalize to the out-of-distribution test data. |
| Outcome: | The proposed method outperforms existing methods that sampled from the training distribution and outperformed existing methods. |
Unsupervised Learning of Sentence Embeddings Using Compositional n-Gram Features (N18-1)
Copied to clipboard
| Challenge: | Currently, unsupervised word embeddings are routinely trained on large amounts of raw text data. |
| Approach: | They propose to use unsupervised word embeddings to train distributed representations of sentences. |
| Outcome: | The proposed method outperforms state-of-the-art models on most benchmark tasks and is robust to the produced general-purpose sentence embeddings. |
Montague semantics and modifier consistency measurement in neural language models (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies on distributional language models have been focused on linguistics and their relationship with semantic formalisms for decades. |
| Approach: | They propose a method for measuring compositional behavior in contemporary language embedding models by introducing three new tests inspired by Montague semantics. |
| Outcome: | The proposed method measures compositional behavior in language embedding models on adjectival modifier phenomena in adjective-noun phrases. |
Static Word Embeddings for Sentence Semantic Representation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to learn fixed-length embeddings for sentence semantics require large computational cost, making it difficult to process billions of sentences cost-efficiently or deploy models on resource-constrained devices such as smartphones. |
| Approach: | They propose to extract word embeddings from a pre-trained Sentence Transformer and improve them with sentence-level principal component analysis followed by knowledge distillation or contrastive learning. |
| Outcome: | The proposed model outperforms existing models on sentence semantic tasks and surpasses a basic Sentence Transformer model (SimCSE) on a text embedding benchmark. |
Building Static Embeddings from Contextual Ones: Is It Useful for Building Distributional Thesauri? (2022.lrec-1)
Copied to clipboard
| Challenge: | contextual language models are dominant in the field of Natural Language Processing, but they are not suitable for all uses. |
| Approach: | They propose a method for building word or type-level embeddings from contextual models . they evaluate a large set of English nouns from the perspective of extracting semantic similarity relations . |
| Outcome: | The proposed method can be used to build word or type embeddings from contextual models . it can be exploited for a wide set of English nouns, showing it can improve distributional thesauri . |