Extracting Conceptual Spaces from LLMs Using Prototype Embeddings (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Conceptual spaces represent entities and concepts using cognitively meaningful dimensions . practical methods for extracting conceptual spaces are currently lacking . |
| Approach: | They propose a strategy in which features are encoded by embedding a description of a corresponding prototype. |
| Outcome: | The proposed approach is highly effective. |
Similar Papers
Interpreting Embedding Spaces by Conceptualization (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models have a significant drawback: they are incomprehensible to humans. |
| Approach: | They propose a method for understanding embeddings by transforming a latent embeddable space into a comprehensible conceptual space. |
| Outcome: | The proposed method compares the semantics of the original latent embedding space to the semantic of the vectors. |
Ranking Entities along Conceptual Space Dimensions with LLMs: An Analysis of Fine-Tuning Strategies (2024.findings-acl)
Copied to clipboard
| Challenge: | Conceptual spaces represent entities in terms of their primitive semantic features. |
| Approach: | They argue that conceptual spaces should be used alongside knowledge graphs in many settings to model entities in terms of their primitive semantic features. |
| Outcome: | The proposed model can rank entities according to a given conceptual space dimension but ground truth rankings for conceptual space dimensions are rare. |
Cabbage Sweeter than Cake? Analysing the Potential of Large Language Models for Learning Conceptual Spaces (2023.emnlp-main)
Copied to clipboard
| Challenge: | Conceptual spaces are constructed from a set of quality dimensions, which are usually learned from human judgements, which means that applications of conceptual spaces are limited to narrow domains. |
| Approach: | They propose to use Large Language Models to learn perceptually grounded representations by comparing them to larger models of the BERT family. |
| Outcome: | The proposed models outperform the largest model, despite being 2 to 3 orders of magnitude smaller. |
AMenDeD: Modelling Concepts by Aligning Mentions, Definitions and Decontextualised Embeddings (2024.lrec-main)
Copied to clipboard
| Challenge: | Contextualised Language Models (LMs) improve on word embeddings by encoding meaning of words in context. |
| Approach: | They propose to learn a unified embedding space in which all three types of representations can be integrated. |
| Outcome: | The proposed model outperforms existing approaches in ontology completion tasks. |
Concept Tokens: Learning Behavioral Embeddings Through Concept Definitions (2026.findings-acl)
Copied to clipboard
| Challenge: | Concept Tokens is a lightweight method that adds a special token to a pretrained LLM . we find that negating the hallucination token reduces hallucines and lowers precision . |
| Approach: | They propose a lightweight method that adds a new special token to a pretrained LLM and learns only its embedding from multiple natural language definitions of a target concept. |
| Outcome: | The proposed method can learn only its embedding from multiple definitions of a target concept . the study shows that it can improve hallucinations and recasting in closed-book questions . |
A Text is Worth Several Tokens: Text Embedding from LLMs Secretly Aligns Well with The Key Tokens (2025.acl-long)
Copied to clipboard
| Challenge: | et al., 2023) show that text embeddings from large language models can be aligned with key tokens in input text. |
| Approach: | They propose a sparse retrieval method based on aligned tokens for large language models . they show that this phenomenon is universal and is not affected by model architecture . |
| Outcome: | The proposed method can achieve 80% of the dense retrieval effect of the same model while reducing the computation significantly. |
Partial Colexifications Improve Concept Embeddings (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for embedding words from colexification networks are limited to the word level, ignoring lexical relations that would only hold for parts of words in a given language. |
| Approach: | They propose to embed concepts from automatically constructed colexification networks . they use lexical similarity ratings and word association data to evaluate the methods . |
| Outcome: | The proposed methods capture and represent different semantic relationships between concepts. |
Adjusting Interpretable Dimensions in Embedding Space with Human Judgments (2024.naacl-long)
Copied to clipboard
| Challenge: | Embedding spaces contain interpretable dimensions indicating gender, formality in style, or even object properties. |
| Approach: | They combine seed-based vectors with human ratings of where words fall along a specific dimension to evaluate on predicting object properties and stylistic properties. |
| Outcome: | The proposed model improves on seed-based vectors and human ratings on object properties and stylistic properties. |
Entity or Relation Embeddings? An Analysis of Encoding Strategies for Relation Extraction (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to relation extraction use concatenating embeddings of head and tail entities . however, such representations capture the types of the entities involved, leading to false positives and confusion between relations involving entities of the same type. |
| Approach: | They propose a model which combines [MASK] embeddings with entity embedds to learn relation embeddations. |
| Outcome: | The proposed model outperforms the state-of-the-art on several benchmarks . it uses a self-supervised pre-training strategy which further improves the results. |
CALE : Concept-Aligned Embeddings for Both Within-Lemma and Inter-Lemma Sense Differentiation (2026.eacl-long)
Copied to clipboard
| Challenge: | Recent work on Word-in-Context fine-tunes models to investigate lexical meaning but only compares occurrences of the same lemma, limiting the range of captured information. |
| Approach: | They propose an extension to Word-in-Context to include inter-words scenarios by using a dataset and several models on a data set. |
| Outcome: | The proposed models provide efficient multi-purpose representations of lexical meaning that reach best performances in the experiments. |