Coloring the Black Box: What Synesthesia Tells Us about Character Embeddings (2021.eacl-main)
Copied to clipboard
| Challenge: | Neural network models are difficult to understand and are considered "black boxes". |
| Approach: | They use grapheme–color synesthesia to study character embeddings in English . they compare graphemes to phonemes to find the most human-like character embeds . |
| Outcome: | The results show that grapheme-to-phoneme conversion results in the most human-like character embeddings. |
Similar Papers
Analyzing the Surprising Variability in Word Embedding Stability Across Languages (2021.emnlp-main)
Copied to clipboard
| Challenge: | Word embeddings are powerful representations that form the foundation of many natural language processing architectures. |
| Approach: | They explore word embedding stability in a wide range of languages to gain insight into their stability. |
| Outcome: | The proposed results provide insights into word embedding stability in English and other languages. |
Interpreting Character Embeddings With Perceptual Representations: The Case of Shape, Sound, and Color (2022.acl-long)
Copied to clipboard
| Challenge: | Character-level information is included in many NLP models, but evaluating the information encoded in character embeddings is an open issue. |
| Approach: | They propose to use shape, sound, and color embeddings to evaluate the information encoded in character representations in five languages to perform cross-lingual analysis. |
| Outcome: | The proposed classifiers evaluate phonological information encoded in character embeddings and LSTM models. |
More Embeddings, Better Sequence Labelers? (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work suggests contextual embeddings improve sequence labeling accuracy . but, there is no definite conclusion on whether concatenating different kinds of embeddables is effective . |
| Approach: | They propose a family of contextual embeddings that improves sequence labeling accuracy . they conduct extensive experiments on 3 tasks over 18 datasets and 8 languages . |
| Outcome: | The proposed family of contextual embeddings improves the accuracy of sequence labelers over non-contextual embedders. |
Gating Mechanisms for Combining Character and Word-level Word Representations: an Empirical Study (N19-3)
Copied to clipboard
| Challenge: | Existing studies show that combining character and word-level representations improves word and sentence representations . however, word-based embeddings do not account for derivational processes resulting in syntactically-similar words with different meanings. |
| Approach: | They propose to combine character and word-level representations to improve word and sentence representations. |
| Outcome: | The proposed method performed well in several word similarity datasets. |
Do Word Embeddings Capture Spelling Variation? (2020.coling-main)
Copied to clipboard
| Challenge: | Using word embeddings, we analyze spelling variation in word embeds trained on Twitter and Reddit data. |
| Approach: | They propose a new perspective on the analysis of word embeddings by focusing on spelling variation. |
| Outcome: | The proposed analysis shows that word embeddings encode spelling variation patterns of various types to some extent, even when trained using the skipgram model. |
Embeddings in Natural Language Processing (2020.coling-tutorials)
Copied to clipboard
| Challenge: | Embeddings have been a key topic of interest in NLP for the past decade . a quick warm-up introduction to NLP and why it is important to have a semantic comprehension of texts . |
| Approach: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and then move to other types of embeddable vectors . |
| Outcome: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and move to other types of embeddable representations . |
A Deeper Look into Dependency-Based Word Embeddings (N18-4)
Copied to clipboard
| Challenge: | Word embeddings trained with dependency contexts excel at different tasks, and enhanced dependencies often improve performance. |
| Approach: | They propose to use dependency-based word embeddings to capture semantic similarity rather than relatedness. |
| Outcome: | The results show that word embeddings trained with Universal and Stanford dependencies excel at different tasks and that enhanced dependencies often improve performance. |
Learning and Evaluating Character Representations in Novels (2022.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in word embeddings have proven successful in learning entity representations from short texts but do not capture full book-level information. |
| Approach: | They propose two novel ways to learn fixed-length vector representations of characters from novels . they use graph neural network-based embeddings from a full corpus-based character network . |
| Outcome: | The proposed methods outperform text-based embeddings in four tasks. |
Interpretable Text Embeddings and Text Similarity Explanation: A Survey (2025.emnlp-main)
Copied to clipboard
| Challenge: | Text embeddings are a fundamental component in many NLP tasks, but their interpretation and explanation remain challenging. |
| Approach: | They propose a framework for interpretable text embeddings and text similarity explanation . they characterize the main ideas, approaches, and trade-offs and discuss lessons learned . |
| Outcome: | The proposed methods are compared with existing models and compare them with existing ones. |
Perceptual Structure in the absence of grounding: the impact of abstractedness and subjectivity in color language for LLMs (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies show that color perception and color language are suitable for empirically studying the problem. |
| Approach: | They propose to quantify alignment between a defined color space and a feature space in a language model by learning a mapping between embedding space and color space. |
| Outcome: | The results show that there is considerable alignment between a defined color space and the feature space defined by a language model. |