Perceptual Structure in the absence of grounding: the impact of abstractedness and subjectivity in color language for LLMs (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies show that color perception and color language are suitable for empirically studying the problem. |
| Approach: | They propose to quantify alignment between a defined color space and a feature space in a language model by learning a mapping between embedding space and color space. |
| Outcome: | The results show that there is considerable alignment between a defined color space and the feature space defined by a language model. |
Similar Papers
Knowledge Supports Visual Language Grounding: A Case Study on Colour Terms (2020.acl-main)
Copied to clipboard
| Challenge: | In human cognition, world knowledge supports the perception of object colours . a lot of recent work in Language & Vision has looked at grounding language in real-world sensory information. |
| Approach: | They propose to integrate visual information and object-specific knowledge via hard-coded or learned fusion to improve visual grounding of colour terms in realistic objects. |
| Outcome: | The proposed models outperform a baseline model that predicts colour terms solely from visual inputs but show interesting differences when predicting atypical colours of so-called colour diagnostic objects. |
Grounding learning of modifier dynamics: An application to color naming (D19-1)
Copied to clipboard
| Challenge: | Existing models for grounding are unable to understand modified color expressions, such as “light blue”. |
| Approach: | They propose a model that learns more complex transformations in RGB space and a hard ensemble model that selects a color space depending on the modifier-color pair. |
| Outcome: | The proposed model performs better in the HSV color space than the state-of-the-art model. |
Visual Grounding Helps Learn Word Meanings in Low-Data Regimes (2024.naacl-long)
Copied to clipboard
| Challenge: | Modern neural language models (LMs) require distinctly un-human-like ways to achieve these results. |
| Approach: | They train a diverse set of LM architectures with and without auxiliary visual supervision on datasets of varying scales. |
| Outcome: | The proposed models exhibit better learning of syntactic categories, lexical relations, semantic features, word similarity and alignment with human neural representations. |
‘Lighter’ Can Still Be Dark: Modeling Comparative Color Descriptions (P18-2)
Copied to clipboard
| Challenge: | Multimodal approaches to object recognition ground adjectives and nouns from text using comparative adjectives. |
| Approach: | They propose a new paradigm of grounding comparative adjectives within the realm of color descriptions by using a vector model. |
| Outcome: | The proposed model generates representations of comparative adjectives with an average accuracy of 0.65 cosine similarity to the desired direction of change. |
Learning Language through Grounding (2025.naacl-tutorial)
Copied to clipboard
| Challenge: | This tutorial provides a historical overview of grounding and discusses its use in computational linguistics and in computational language processing. |
| Approach: | They introduce the concept of grounding and discuss future directions and open challenges . they will delve into recent progress in learning lexical semantics, syntax, and complex meanings through various forms of ground. |
| Outcome: | This course will provide an overview of the field of grounding and discuss future directions and challenges related to large language models and scaling. |
Multimodal Grounding for Language Processing (C18-1)
Copied to clipboard
| Challenge: | Recent developments in multimodal processing facilitate conceptual grounding of language. |
| Approach: | They analyze multimodal processing to examine the benefits and challenges of multimodal grounding . they focus on multimodal linguistic grounding of verbs which play a crucial role in compositional power of language. |
| Outcome: | The proposed methods improve the cognitive models of human information processing and address the challenges that arise. |
Does Visual Grounding Enhance the Understanding of Embodied Knowledge in Large Language Models? (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Despite significant progress in multimodal language models, it remains unclear whether visual grounding enhances their understanding of embodied knowledge compared to text-only models. |
| Approach: | They propose to assess vision-language models’ perceptual abilities across different sensory modalities through vector comparison and question-answering tasks with over 1,700 questions. |
| Outcome: | The proposed benchmark assesses the models’ perceptual abilities across different sensory modalities through vector comparison and question-answering tasks with over 1,700 questions. |
Incorporating Visual Semantics into Sentence Representations within a Grounded Space (D19-1)
Copied to clipboard
| Challenge: | Language grounding is an active field aiming at enriching textual representations with visual information. |
| Approach: | They propose to transfer visual information to textual representations by learning an intermediate representation space: the grounded space. |
| Outcome: | The proposed model outperforms the previous state-of-the-art on classification and semantic relatedness tasks. |
Language in a (Search) Box: Grounding Language Learning in Real-World Human-Machine Interaction (2021.naacl-main)
Copied to clipboard
| Challenge: | Scholarly work in this area uses toy worlds and synthetic linguistic data, but grounded language learning offers several practical and scientific advantages. |
| Approach: | They propose to model teacher-learner dynamics through natural interactions occurring between users and search engines. |
| Outcome: | The proposed model is better than non-grounded models on compositionality and zero-shot inference tasks. |
How Well Do Large Language Models Truly Ground? (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing research defines “grounding” as having the correct answer, which does not ensure the reliability of the entire response. |
| Approach: | They propose a stricter definition of grounding: fully utilizes the necessary knowledge from the provided context and stays within the limits of that knowledge. |
| Outcome: | The proposed model can be ground on external contexts and maintain its correct answer. |