Papers with Word2Vec
Embeddings in Natural Language Processing (2020.coling-tutorials)
Copied to clipboard
| Challenge: | Embeddings have been a key topic of interest in NLP for the past decade . a quick warm-up introduction to NLP and why it is important to have a semantic comprehension of texts . |
| Approach: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and then move to other types of embeddable vectors . |
| Outcome: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and move to other types of embeddable representations . |
Using Graphs for Word Embedding with Enhanced Semantic Relations (D19-53)
Copied to clipboard
| Challenge: | Word embedding algorithms are based on sequential text input, while others are utilizing a graph representation of text. |
| Approach: | They propose a word embedding algorithm based on a directed word graph to provide additional information for sequential text input algorithms. |
| Outcome: | The proposed algorithm is based on a directed word graph to provide additional information for sequential text input algorithms. |
Domain-Specific Word Embeddings with Structure Prediction (2023.tacl-1)
Copied to clipboard
| Challenge: | Current word embedding methods do not provide a way to use or predict information on structure between sub-corpora, time or domain. |
| Approach: | They propose a word embedding method that provides general word representations for the whole corpus, domain-specific representations and embeddable alignment simultaneously. |
| Outcome: | The proposed method provides better performance than baselines on a dataset of science and philosophy articles. |
Detecting Omissions in LLM-Generated Medical Summaries (2025.emnlp-industry)
Copied to clipboard
Achir Oukelmoun, Nasredine Semmar, Gaël de Chalendar, Clement Cormi, Mariame Oukelmoun, Eric Vibert, Marc-Antoine Allard
| Challenge: | Large Language Models (LLMs) have created a number of use cases in the medical field . omissions in summaries can jeopardize the decision-making process . |
| Approach: | They propose a dataset to evaluate omissions in large-scale medical summaries . they propose 'embedKDECheck' method that uses embeddings generated by a third-party NLP model . |
| Outcome: | The proposed method is well-suited for resource-constrained environments. |
No, you’re not alone: A better way to find people with similar experiences on Reddit (D19-55)
Copied to clipboard
| Challenge: | a probabilistic clustering algorithm can help users find posts that discuss experiences similar to their own . a recent study shows that probabilistic Clustering can yield a better performance than baseline clustering methods . |
| Approach: | They propose a probabilistic clustering algorithm that can help Reddit users find posts that discuss experiences similar to their own. |
| Outcome: | The proposed algorithm can find posts that discuss experiences similar to their own . it performs better than baseline clustering methods due to high runtime overhead . |
Tsetlin Machine Embedding: Representing Words Using Logical Expressions (2024.findings-eacl)
Copied to clipboard
| Challenge: | Embedding words in vector space is a fundamental first step in state-of-the-art natural language processing. |
| Approach: | They propose to embed words in vector space using propositional logic instead of dense vectors . they evaluate embeddings on intrinsic and extrinsic benchmarks and visualize word clusters based on their results . |
| Outcome: | The proposed model outperforms GLoVe on six classification tasks. |
Word Embedding Binarization with Semantic Information Preservation (2020.coling-main)
Copied to clipboard
| Challenge: | Word and Sentence embeddings are one of the most common starting points of any NLP task. |
| Approach: | They propose a way to convert word embedding to binary representation to reduce overall size . they propose different approaches suitable for different downstream tasks based on contextual and semantic information. |
| Outcome: | The proposed method reduces the size of the embedding while keeping the semantic and relational knowledge intact. |
Inexpensive Domain Adaptation of Pretrained Language Models: Case Studies on Biomedical NER and Covid-19 QA (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained Language Models (PTLMs) are typically pretraining on target-domain text, which is expensive in terms of hardware, runtime and CO 2 emissions. |
| Approach: | They propose a faster, CPU-only domainadaptation method that trains Word2Vec on target-domain text and aligns the resulting word vectors with the wordpiece vectors of a general-domain PTLM. |
| Outcome: | The proposed method covers 60% of the BioBERT - BERT F1 delta, 5% of BioBERTS’s CO2 footprint and 2% of its cloud compute cost. |
Word Embedding Evaluation for Sinhala (2020.lrec-1)
Copied to clipboard
| Challenge: | Word embeddings are a key component of the natural language processing process. |
| Approach: | They evaluate three standard word embedding models for Sinhala language using two evaluation methods: intrinsic evaluation and extrinsic evaluation. |
| Outcome: | The proposed models performed best in the three evaluation tasks, while FastText and Glove showed the lowest accuracies. |
MGAD: Multilingual Generation of Analogy Datasets (L18-1)
Copied to clipboard
| Challenge: | Existing methods for word embedding evaluation are computationally expensive and task-specific. |
| Approach: | They propose a minimally supervised method for generating word embedding evaluation datasets for a large number of languages using existing dependency treebanks and parsers. |
| Outcome: | The proposed method evaluates three popular word embedding algorithms against these datasets and shows that their performance varies between syntactic categories. |
The Effects of Unimodal Representation Choices on Multimodal Learning (L18-1)
Copied to clipboard
| Challenge: | In the real world, multiple modes of information are gathered to create knowledge in a way humans can understand. |
| Approach: | They propose to combine unimodal representations to map multiple modes of information to a single space . they argue that the way they are combined can affect performance and classification metrics . |
| Outcome: | The proposed model can be used to correlate words in a textual description of an object with multimodal representations. |
Towards end-2-end learning for predicting behavior codes from spoken utterances in psychotherapy conversations (2020.acl-main)
Copied to clipboard
| Challenge: | Xu and Sarikaya, 2014) proposes a framework for predicting utterance level labels directly from speech features. |
| Approach: | They propose a framework for predicting utterance level labels directly from speech features using a pretrained Speech-2-Vector encoder as bottleneck. |
| Outcome: | The proposed model outperforms state-of-the-art approaches which use transcribed text for the task of predicting psychotherapy-relevant behavior codes. |
Representing the Toddler Lexicon: Do the Corpus and Semantics Matter? (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing studies on child language development have relied on adult-based measures to model their lexicons. |
| Approach: | They propose to use transcripts of child-directed conversations, picture books and dialog from G-rated movies to approximate the language input a North American preschooler might hear. |
| Outcome: | The proposed model outperforms models based on the existing corpus and the existing model. |
Interpreting Pretrained Contextualized Representations via Reductions to Static Embeddings (2020.acl-main)
Copied to clipboard
| Challenge: | Contextualized representations have become the default for downstream NLP applications. |
| Approach: | They propose a method for converting from contextualized representations to static lookup-table embeddings and apply it to 5 popular pretrained models and 9 sets of pretrained weights. |
| Outcome: | The proposed methods show that pooling over many contexts significantly improves representational quality under intrinsic evaluation. |
A Neural Network Model for Part-Of-Speech Tagging of Social Media Texts (L18-1)
Copied to clipboard
| Challenge: | Recent approaches based on end-to-end Deep Neural Networks (DNNs) have shown promising results for Natural Language Processing (NLP). |
| Approach: | They propose a neural network model for part-of-speech (POS) tagging of User-Generated Content (UGC) such as Twitter, Facebook and Web forums that uses character and word representations. |
| Outcome: | The proposed model is end-to-end and uses character and word representations . it is compared with existing models on social media in English, german, french, italian and spanish . |
Using Context-to-Vector with Graph Retrofitting to Improve Word Embeddings (2022.acl-long)
Copied to clipboard
| Challenge: | Contextualized embeddings are expensive and resource-demanding, hence environmentally unfriendly. |
| Approach: | They propose a method to convert contextualized embeddings from pre-trained models into static embeddables using synonym knowledge and weighted vector distribution. |
| Outcome: | The proposed method outperforms baseline embeddings by a large margin through extrinsic and intrinsic tasks. |