A Method for Studying Semantic Construal in Grammatical Constructions with Interpretable Contextual Embedding Spaces (2023.acl-long)
Copied to clipboard
| Challenge: | Existing paradigms for the linguistically oriented exploration of large neural language models include treating the model as a linguistic test subject by measuring output on test sentences and building probing classifiers on top of embeddings to test whether the embeddables are sensitive to certain properties like dependency structure. |
| Approach: | They project contextual embeddings into interpretable semantic spaces, each defined by a different set of psycholinguistic feature norms. |
| Outcome: | The proposed method can probe the distributional meaning of syntactic constructions at a templatic level, abstracted away from specific lexemes. |
Similar Papers
A Survey on Automatically-Constructed WordNets and their Evaluation: Lexical and Word Embedding-based Approaches (L18-1)
Copied to clipboard
| Challenge: | WordNets are lexical databases in which groups of synonyms are stored according to the semantic relationships between them. |
| Approach: | This paper describes various approaches to constructing WordNets automatically by leveraging traditional lexical resources and newer trends such as word embeddings. |
| Outcome: | The proposed methods leverage traditional lexical resources and newer trends such as word embeddings to build and evaluate WordNets. |
Interpreting Embedding Spaces by Conceptualization (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models have a significant drawback: they are incomprehensible to humans. |
| Approach: | They propose a method for understanding embeddings by transforming a latent embeddable space into a comprehensible conceptual space. |
| Outcome: | The proposed method compares the semantics of the original latent embedding space to the semantic of the vectors. |
Exploring Category Structure with Contextual Language Models and Lexical Semantic Networks (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent work on word embeddings reports low correlations with human ratings . contextual language models (CLMs) have been successful in acquiring semantic and world knowledge. |
| Approach: | They propose to use BERT to probe contextual language models for predicting typicality scores. |
| Outcome: | The proposed methods improve on previous studies on word embeddings and their ability to predict typicality scores. |
Analyzing the Surprising Variability in Word Embedding Stability Across Languages (2021.emnlp-main)
Copied to clipboard
| Challenge: | Word embeddings are powerful representations that form the foundation of many natural language processing architectures. |
| Approach: | They explore word embedding stability in a wide range of languages to gain insight into their stability. |
| Outcome: | The proposed results provide insights into word embedding stability in English and other languages. |
How Well Do Text Embedding Models Understand Syntax? (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing text embedding models have not addressed syntactic understanding challenges, highlighting ineffectiveness and enhancing generalization ability. |
| Approach: | They propose to examine the ability of text embedding models to generalize across syntactic contexts. |
| Outcome: | The proposed models exhibit high similarity socres at this simple task. |
Constructions are Revealed in Word Distributions (2025.emnlp-main)
Copied to clipboard
| Challenge: | Construction grammar posits that constructions are form-meaning pairings that are acquired through experience with language. |
| Approach: | They propose to use a RoBERTa model to model constructions as patterns of statistical affinity . they show that statistical affinity is likely an important, but partial, signal available to learners . |
| Outcome: | The proposed model shows that constructions will be revealed as patterns of statistical affinity . the proposed model is based on a model that is able to distinguish constructions from text . |
Building Static Embeddings from Contextual Ones: Is It Useful for Building Distributional Thesauri? (2022.lrec-1)
Copied to clipboard
| Challenge: | contextual language models are dominant in the field of Natural Language Processing, but they are not suitable for all uses. |
| Approach: | They propose a method for building word or type-level embeddings from contextual models . they evaluate a large set of English nouns from the perspective of extracting semantic similarity relations . |
| Outcome: | The proposed method can be used to build word or type embeddings from contextual models . it can be exploited for a wide set of English nouns, showing it can improve distributional thesauri . |
Semantic Geometry of Sentence Embeddings (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Sentence embeddings are central to natural language processing, but their internal features are not interpretable and users lack fine-grained control for downstream tasks. |
| Approach: | They propose a formal framework to characterize the organization of features in sentence embeddings . they show how they can be composed to capture richer semantic structures . |
| Outcome: | The proposed method can be used to capture richer semantic structures. |
Interpretable Text Embeddings and Text Similarity Explanation: A Survey (2025.emnlp-main)
Copied to clipboard
| Challenge: | Text embeddings are a fundamental component in many NLP tasks, but their interpretation and explanation remain challenging. |
| Approach: | They propose a framework for interpretable text embeddings and text similarity explanation . they characterize the main ideas, approaches, and trade-offs and discuss lessons learned . |
| Outcome: | The proposed methods are compared with existing models and compare them with existing ones. |
Incorporating Contextual and Syntactic Structures Improves Semantic Similarity Modeling (D19-1)
Copied to clipboard
| Challenge: | Semantic similarity modeling is central to many NLP problems such as question answering. |
| Approach: | They propose a pairwise word interaction model with syntactic structure priors to explore their effectiveness. |
| Outcome: | Extensive evaluations on eight benchmark datasets show that incorporating structural information improves over strong baselines. |