Bhargav Srinivasa Desikan, Tasker Hull, Ethan Nadler, Douglas Guilbeault, Aabir Abubakar Kar, Mark Chu, Donald Ruggiero Lo Sardo
| Challenge: | Existing approaches to natural language processing ignore embodied sensory aspects of language. |
| Approach: | They propose a Python package that provides word embeddings based on Google Image search results. |
| Outcome: | The proposed package provides word embeddings based on the color distributions of Google Image search results. |
Similar Papers
A Probabilistic Model for Joint Learning of Word Embeddings from Texts and Images (D18-1)
Copied to clipboard
| Challenge: | Existing approaches combine language and perception to infer word embeddings . however, the embeddables produced by such models do not reflect the actual word representations. |
| Approach: | They propose a probabilistic model that integrates linguistic and perceptual inputs to explain observed word-context pairs in a text corpus. |
| Outcome: | The proposed model achieves competitive or stronger results on tasks of assessing pairwise word similarity and image/caption retrieval compared to other state-of-the-art models. |
Knowledge Supports Visual Language Grounding: A Case Study on Colour Terms (2020.acl-main)
Copied to clipboard
| Challenge: | In human cognition, world knowledge supports the perception of object colours . a lot of recent work in Language & Vision has looked at grounding language in real-world sensory information. |
| Approach: | They propose to integrate visual information and object-specific knowledge via hard-coded or learned fusion to improve visual grounding of colour terms in realistic objects. |
| Outcome: | The proposed models outperform a baseline model that predicts colour terms solely from visual inputs but show interesting differences when predicting atypical colours of so-called colour diagnostic objects. |
Word2Sense: Sparse Interpretable Word Embeddings (P19-1)
Copied to clipboard
| Challenge: | Word2Sense embeddings are interpretable, but they are sparse and fast to compute . a unitary rotation can be applied to many of these embeddables retaining their utility for computational tasks while changing the values of individual coordinates. |
| Approach: | They propose an unsupervised method to generate Word2Sense word embeddings that are interpretable. |
| Outcome: | The proposed method compares well with other unsupervised word embeddings on NLP tasks. |
Perceptual Structure in the absence of grounding: the impact of abstractedness and subjectivity in color language for LLMs (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies show that color perception and color language are suitable for empirically studying the problem. |
| Approach: | They propose to quantify alignment between a defined color space and a feature space in a language model by learning a mapping between embedding space and color space. |
| Outcome: | The results show that there is considerable alignment between a defined color space and the feature space defined by a language model. |
Uncovering Visual-Semantic Psycholinguistic Properties from the Distributional Structure of Text Embedding Space (2025.acl-long)
Copied to clipboard
| Challenge: | Imageability and concreteness are psycholinguistic properties that link visual and semantic spaces. |
| Approach: | They propose an unsupervised measure that quantifies sharpness of peaks in an image-caption dataset. |
| Outcome: | The proposed method is more robust than existing methods and predicts these properties for classification. |
Advances in Pre-Training Distributed Word Representations (L18-1)
Copied to clipboard
| Challenge: | Pre-trained word representations are a building block of many Natural Language Processing and Machine Learning applications. |
| Approach: | They propose to combine known tricks and a set of publicly available pre-trained word vector representations to train high-quality representations. |
| Outcome: | The proposed models outperform the current state of the art on a number of tasks while maintaining a high training speed to scale to massive amount of data. |
‘Lighter’ Can Still Be Dark: Modeling Comparative Color Descriptions (P18-2)
Copied to clipboard
| Challenge: | Multimodal approaches to object recognition ground adjectives and nouns from text using comparative adjectives. |
| Approach: | They propose a new paradigm of grounding comparative adjectives within the realm of color descriptions by using a vector model. |
| Outcome: | The proposed model generates representations of comparative adjectives with an average accuracy of 0.65 cosine similarity to the desired direction of change. |
PBoS: Probabilistic Bag-of-Subwords for Generalizing Word Embedding (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing word embeddings assume fixed finite-size vocabularies, hindering their ability to provide useful word representations for out-of-vocaulary words. |
| Approach: | They propose a model that generalizes word embeddings without extra contextual information . they use the spellings of words to model subword segmentation and compute subword-based compositional word embeds. |
| Outcome: | The proposed model can generate meaningful subword segmentations without any source of explicit morphological knowledge. |
Exploring Category Structure with Contextual Language Models and Lexical Semantic Networks (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent work on word embeddings reports low correlations with human ratings . contextual language models (CLMs) have been successful in acquiring semantic and world knowledge. |
| Approach: | They propose to use BERT to probe contextual language models for predicting typicality scores. |
| Outcome: | The proposed methods improve on previous studies on word embeddings and their ability to predict typicality scores. |
Visual Grounding Helps Learn Word Meanings in Low-Data Regimes (2024.naacl-long)
Copied to clipboard
| Challenge: | Modern neural language models (LMs) require distinctly un-human-like ways to achieve these results. |
| Approach: | They train a diverse set of LM architectures with and without auxiliary visual supervision on datasets of varying scales. |
| Outcome: | The proposed models exhibit better learning of syntactic categories, lexical relations, semantic features, word similarity and alignment with human neural representations. |