Measuring Idiomaticity in Text Embedding Models with epsilon-compositionality (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing studies on compositionality of text embedding models have limited understanding of the principle . idioms have traditionally been seen as non-compositional . |
| Approach: | They propose to use formal definitions to define compositionality in text embedding models . they find that most models differentiate between idiomatic and non-idiomatic phrases . |
| Outcome: | The proposed model is able to differentiate between idiomatic and non-idiomatic phrases, the authors show . |
Similar Papers
Montague semantics and modifier consistency measurement in neural language models (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies on distributional language models have been focused on linguistics and their relationship with semantic formalisms for decades. |
| Approach: | They propose a method for measuring compositional behavior in contemporary language embedding models by introducing three new tests inspired by Montague semantics. |
| Outcome: | The proposed method measures compositional behavior in language embedding models on adjectival modifier phenomena in adjective-noun phrases. |
Quantifying Compositionality of Classic and State-of-the-Art Embeddings (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Static word embeddings make strong claims about compositionality, but the SOTA generative models go too far in the other direction. |
| Approach: | a new study evaluates the compositionality of word embeddings by canonical correlation analysis . strong compositional signals are observed in later training stages across data modalities . |
| Outcome: | a new evaluation of compositional models shows that they exploit access meanings when justified . strong compositional signals are observed in later training stages and in deeper layers of the transformer-based model before a decline at the top layer. |
How do Transformer Embeddings Represent Compositions? A Functional Analysis (2025.findings-acl)
Copied to clipboard
| Challenge: | Despite the popularity of transformer-based models, little is known about how they represent compound words and whether they are compositional. |
| Approach: | They evaluate compositionality in mistral, OpenAI Large, and Google embedding models and compare them with BERT. |
| Outcome: | The proposed models perform best in addition, multiplication, dilation, regression, and the classic vector addition model performs almost as well as any other model. |
Additive Compositionality of Word Vectors (D19-55)
Copied to clipboard
| Challenge: | Existing research on justifying additive compositionality of word embedding models requires a rather strong assumption of uniform word distribution. |
| Approach: | They propose to relax the assumption of uniform word distribution and propose more realistic conditions for proving additive compositionality. |
| Outcome: | The proposed model improves on word similarity and noisy sentence similarity. |
Non-Compositionality in Sentiment: New Data and Analyses (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Many studies on sentiment analysis focus on the fact that sentiment computations are compositional . linguistic utterances often do not adhere to strict patterns and can be surprising when looking at the individual words involved. |
| Approach: | They propose a method for obtaining non-compositionality ratings for phrases with respect to their sentiment . they also propose evaluating computational models for sentiment analysis using the rating resource . |
| Outcome: | The proposed method enables non-compositional ratings for phrases with respect to their sentiment . the results are compared with a new resource of ratings for 259 phrases . |
Unified Representation for Non-compositional and Compositional Expressions (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing representations of non-compositional language are based on BART, but they are not as accurate as the state-of-the-art IE representation model, GIEA. |
| Approach: | They propose a language model, PIER+, that builds on BART and can generate semantically meaningful and contextually appropriate representations for English potentially idiomatic expressions. |
| Outcome: | The proposed model achieves 33% higher homogeneity score on embedding clustering than BART, while sacrificing performance on NLU tasks (+/- 1% accuracy) |
Getting BART to Ride the Idiomatic Train: Learning to Represent Idiomatic Expressions (2022.tacl-1)
Copied to clipboard
| Challenge: | Prior work has identified deficiencies in their contextualized representation stemming from the underlying compositional paradigm of representation. |
| Approach: | They propose to use an adapter as a lightweight non-compositional language expert trained on idiomatic sentences to build idiomity into BART. |
| Outcome: | The proposed approach improves idiomaticity over baselines and up to 25% higher sequence accuracy on idiom processing tasks. |
Probing for idiomaticity in vector space models (2021.eacl-main)
Copied to clipboard
| Challenge: | Contextualised word representation models are used to represent idiomaticity in language. |
| Approach: | They propose probing measures to assess if some of the expected linguistic properties of noun compounds are readily available in some standard and widely used representations. |
| Outcome: | The proposed models show that idiomaticity is not yet accurately represented by contextualised models. |
On the Compositionality Prediction of Noun Phrases using Poincaré Embeddings (P19-1)
Copied to clipboard
| Challenge: | idiomatic phrases have a non-compositional meaning, meanings of which can be derived from constituents and their grammatical relations. |
| Approach: | They propose to combine hierarchical and distributional information to blend hierarchic and distribution-based hierarchies to detect compositionality for noun phrases. |
| Outcome: | The proposed technique achieves significant improvements over state-of-the-art models based on distributional information and a weighted average of the distributional similarity and p-like function. |
Syntax-guided Neural Module Distillation to Probe Compositionality in Sentence Embeddings (2023.eacl-main)
Copied to clipboard
| Challenge: | Past work on sentence embedding models faces issues determining the causal impact of implicit syntax representations. |
| Approach: | They construct a neural module net based on a transformer model and train it end-to-end to approximate the sentence’s embedding. |
| Outcome: | The proposed model captures whether syntax is a strong model of its compositional ability. |