Papers with Embeddings
Embeddings in Natural Language Processing (2020.coling-tutorials)
Copied to clipboard
| Challenge: | Embeddings have been a key topic of interest in NLP for the past decade . a quick warm-up introduction to NLP and why it is important to have a semantic comprehension of texts . |
| Approach: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and then move to other types of embeddable vectors . |
| Outcome: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and move to other types of embeddable representations . |
Neural Token Representations and Negation and Speculation Scope Detection in Biomedical and General Domain Text (D19-62)
Copied to clipboard
| Challenge: | Existing evidence for improved performance on natural language tasks is unclear to what degree the learned token representations capture and encode highlevel morphological/syntactic knowledge about the usage of a given token in a sentence. |
| Approach: | They propose to use context-aware token representations to capture morphological/syntactic knowledge about the usage of a given word/token in a sentence. |
| Outcome: | The proposed representations capture and encode high-level morphological/syntactic knowledge about the usage of a given token in a sentence. |
Parallax: Visualizing and Understanding the Semantics of Embedding Spaces via Algebraic Formulae (P19-3)
Copied to clipboard
| Challenge: | Embeddings are a fundamental component of many modern machine learning and natural language processing models. |
| Approach: | They propose a tool for visualizing embedding spaces using parametric projections . they demonstrate the power of Parallax and propose % task-oriented approach . |
| Outcome: | The proposed tool is based on two-dimensional projections without interpretable semantics . it enhances interpretability and allows for more fine-grained analysis . |
Cross-lingual Knowledge Graph Alignment via Graph Convolutional Networks (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to align multilingual knowledge graphs with counterparts in different languages are not effective. |
| Approach: | They propose a novel approach for cross-lingual KG alignment via graph convolutional networks . they train GCNs to embed entities of each language into a unified vector space . |
| Outcome: | The proposed approach gets the best performance on real multilingual KGs compared with other embedding-based approaches. |
Interpretable Company Similarity with Sparse Autoencoders (2025.acl-industry)
Copied to clipboard
Marco Molinari, Victor Shao, Luca Imeneo, Mateusz Mikolajczak, Abhimanyu Pandey, Vladimir Tregubiak, Sebastião Kuznetsov Ryder Torres Pereira
| Challenge: | Traditionally, company comparisons rely on relative returns and discrete classifications, or a combination of both. |
| Approach: | They propose to use clusters of embeddings to enhance the interpretability of Large Language Models by decomposing Large Language models activations into interpretable features. |
| Outcome: | The proposed clusters of embeddings capture the internal representation of a company description, rather than just semantic similarity alone. |
Robust and Minimally Invasive Watermarking for EaaS (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing watermarking methods use a target embedding to create watermarks, but this method results in each embeddable having the same component, making it difficult to remove the watermark. |
| Approach: | They propose to use embedding watermarks to protect EaaS from model extraction attacks . eaas is vulnerable to model extraction, highlighting the need for copyright protection . |
| Outcome: | The proposed method can watermark embeddings against model extraction attacks without sacrificing the quality of the embeddables. |
Memory, Show the Way: Memory Based Few Shot Word Representation Learning (D18-1)
Copied to clipboard
| Challenge: | Existing word embedding methods for distributed semantic models require limited examples to learn a high quality representation. |
| Approach: | They propose a memory-based embedding learning method capable of acquiring word representations from limited context. |
| Outcome: | The proposed method delivers impressive performance on two challenging few-shot word similarity tasks. |
Privacy-Preserving Text Classification on BERT Embeddings with Homomorphic Encryption (2022.naacl-main)
Copied to clipboard
| Challenge: | Embeddings compress information into low-dimensional vectors, but can leak private information about sensitive attributes of text. |
| Approach: | They propose a method to privatize embeddings based on homomorphic encryption to prevent leakage of sensitive information in the process of text classification. |
| Outcome: | The proposed method can protect embeddings from leakage while preserving their utility on downstream tasks. |
Text Embedding Inversion Security for Multilingual Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | storing sensitive information as embeddings is susceptible to security breaches, as text can be reconstructed from embeddables . study explores multilingual inversion attacks using a masking defense . |
| Approach: | They propose a simple masking defense that can be used to decode embedded text . they define the problem of black-box multilingual and crosslingual inversion attacks . |
| Outcome: | The proposed defense is effective for both monolingual and multilingual models. |
From Prejudice to Parity: A New Approach to Debiasing Large Language Model Word Embeddings (2025.coling-main)
Copied to clipboard
| Challenge: | Existing work in this field has looked most commonly into gender bias, racial bias, and religious bias. |
| Approach: | They propose an algorithm that uses a neural network to perform ‘soft debiasing’ and build on the seminal work of (CITATION) and (CitATION). |
| Outcome: | The proposed algorithm outperforms current methods on gender, race, and religion metrics on a wide range of metrics. |
Multilingual Culture-Independent Word Analogy Datasets (2020.lrec-1)
Copied to clipboard
| Challenge: | In text processing, deep neural networks use word embeddings as an input. |
| Approach: | They propose to use benchmark datasets to compare the quality of word embeddings in text processing . they use a word analogy task in Croatian, English, Estonian, Finnish, Latvian, Lithuanian, Russian, Slovenian, and Swedish . |
| Outcome: | The proposed datasets are culturally independent and cross-lingual for the languages used. |
Search-Adaptor: Embedding Customization for Information Retrieval (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods to embed text in large language models are limited to zero-shot setups and can be integrated with any LLM. |
| Approach: | They propose a method for customizing LLMs for information retrieval by modifying the embeddings generated by pre-trained LLM models and can be integrated with any LLM. |
| Outcome: | The proposed method improves performance on English, multilingual, and multimodal retrieval datasets by 5% over 14 BEIR datasets. |
Contextualized Embeddings based Transformer Encoder for Sentence Similarity Modeling in Answer Selection Task (2020.lrec-1)
Copied to clipboard
| Challenge: | Word embeddings that consider context have attracted great attention for natural language processing tasks in recent years. |
| Approach: | They propose two different approaches to integrate contextualized word embeddings with transformer encoders for sentence similarity modeling. |
| Outcome: | The proposed model outperforms the feature-based approach on six datasets. |
Beyond the Average Reader: the Reader Embedding Approach (2025.findings-acl)
Copied to clipboard
| Challenge: | a new approach to predict reading times is proposed to use eye-tracking data to collect data from all subjects rather than from the most similar ones. |
| Approach: | They propose a method to collect eye-tracking data that are averaged and used to train learning models. |
| Outcome: | The proposed approach outperforms existing methods by combining eye-tracking data with averaged data. |
GASE: Graph-Aware Semantic Embedding Learning with Frozen LLMs for Text-Attributed Graphs (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown strong potential for text-attributed graph (TAG) learning, yet effectively integrating LLM semantics with graph structural information remains challenging. |
| Approach: | They propose a framework for learning Graph-Aware Semantic Embeddings using frozen LLMs. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on node classification and achieves a 5 speedup over fine-tuning-based methods. |