EMBEDDING

Obligation and Prohibition Extraction Using Hierarchical RNNs (P18-2)

Challenge:	Several pre-training models of different modalities are showing a rising trend of homogeneity in their model structures.
Approach:	They propose a toolkit that supports pre-training models of different modalities.
Outcome:	The proposed toolkit can match the performance of the original implementations on text, vision, and audio benchmarks.

Copied to clipboard

Ilias Chalkidis, Ion Androutsopoulos, Achilleas Michos

Improving Knowledge Graph Embedding Using Affine Transformations of Entities Corresponding to Each Relation (2021.findings-emnlp)

Challenge:	Existing methods for contract element extraction and contract element classification focus on indicative tokens, but they are not as efficient as the current ones.
Approach:	They propose a self-attention mechanism that converts each sentence to an embedding and processes the embeddables to classify each sentence.
Outcome:	The proposed method outperforms the flat BILSTM classifier even when it considers surrounding sentences because it has a broader discourse view.

Copied to clipboard

Jinfa Yang, Yongjie Shi, Xin Tong, Robin Wang, Taiyan Chen, Xianghua Ying

A Biologically Plausible Parser (2021.tacl-1)

Challenge:	Existing knowledge graph embedding methods use k-dimensional vectors to represent each entity in a knowledge graph.
Approach:	They propose to use affine transformations to embed knowledge graphs using previous methods . they propose to add k additional variables to the existing methods to perform embedding .
Outcome:	The proposed method outperforms RotatE, Distmult and ComplEx on various data sets.

Copied to clipboard

Daniel Mitropolsky, Michael J. Collins, Christos H. Papadimitriou

Challenge:	a recent computational framework for cognitive function is being proposed to model brain activity . a parser of English is capable of parsing nontrivial sentences, but it can be extended to many directions .
Approach:	They propose a parser of English effectuated by biologically plausible neurons and synapses . they propose recursion, embedding, and polysemy as ways to parse nontrivial sentences .
Outcome:	The proposed parser can handle recursion, embedding, and polysemy in nontrivial sentences . the proposed framework can be extended to many directions encompassing much of language .

Unified Multimodal Interleaved Document Representation for Retrieval (2026.findings-eacl)

Copied to clipboard

Jaewoo Lee, Joonho Ko, Jinheon Baek, Soyeong Jeong, Sung Ju Hwang

Finnish Hate-Speech Detection on Social Media Using CNN and FinBERT (2022.lrec-1)

Challenge:	Existing methods focus on textual content, ignoring the fact that documents can contain multiple modalities.
Approach:	They propose a method that holistically embeds documents interleaved with multiple modalities . they use vision-language models that combine text, images, and tables into a unified format .
Outcome:	The proposed method outperforms baselines on textual and multimodal queries.

Copied to clipboard

Md Saroar Jahan, Mourad Oussalah, Nabil Arhab

Language Models

Challenge:	Existing tools to identify hate posts from social media are limited in the field of online hate speech detection.
Approach:	They propose to use finBERT to generate a Finnish hate speech dataset . finBERt has a 91.7% accuracy and 90.8% F1 score value, they say .
Outcome:	The proposed model outperforms state-of-the-art models in Finnish and other languages.

Robust and Minimally Invasive Watermarking for EaaS (2025.findings-acl)

Copied to clipboard

Zongqi Wang, Baoyuan Wu, Jingyuan Deng, Yujiu Yang

Robustness in NLP

Challenge:	Existing watermarking methods use a target embedding to create watermarks, but this method results in each embeddable having the same component, making it difficult to remove the watermark.
Approach:	They propose to use embedding watermarks to protect EaaS from model extraction attacks . eaas is vulnerable to model extraction, highlighting the need for copyright protection .
Outcome:	The proposed method can watermark embeddings against model extraction attacks without sacrificing the quality of the embeddables.

Global Textual Relation Embedding for Relational Understanding (P19-1)

Copied to clipboard

Zhiyu Chen, Hanwen Zha, Honglei Liu, Wenhu Chen, Xifeng Yan, Yu Su

Compressing then Matching: An Efficient Pre-training Paradigm for Multimodal Embedding (2026.acl-long)

Challenge:	Existing methods to learn textual relation embeddings are lacking in large open-domain corpora.
Approach:	They propose to learn a general-purpose embedding of textual relations using a large dataset from Freebase.
Outcome:	The proposed embedding can facilitate downstream tasks requiring relational understanding of the text.

Copied to clipboard

Da Li, Yuxiao Luo, Keping Bi, Jiafeng Guo, Wei Yuan, Biao Yang, Yan Wang, Fan Yang, Tingting Gao, Guorui Zhou

DWA-KD: Dual-Space Weighting and Time-Warped Alignment for Cross-Tokenizer Knowledge Distillation (2026.findings-eacl)

Challenge:	Recent approaches demonstrate that MLLMs can be adapted into competitive embedding models via large-scale contrastive learning.
Approach:	They propose a compressed pre-training phase which serves as a warm-up stage for contrastive learning.
Outcome:	The proposed model achieves state-of-the-art among MLLMs of comparable size on the MMEB, realizing optimization in both efficiency and effectiveness.

Copied to clipboard

Duc Trung Vu, Pham Khanh Chi, Dat Phi Van, Linh Ngo Van, Dinh Viet Sang, Trung Le

Training LLMs to be Better Text Embedders through Bidirectional Reconstruction (2025.emnlp-main)

Challenge:	Existing cross-tokenizer distillation methods are limited by suboptimal alignment across sequence and vocabulary levels.
Approach:	They propose a cross-tokenizer distillation framework that enhances token-wise distillation . they use dual-space entropy-based weighting to achieve precise sequence-level alignment .
Outcome:	The proposed framework outperforms state-of-the-art methods in large language models but has high computational and memory costs.

Copied to clipboard

Chang Su, Dengliang Shi, Siyuan Huang, Jintao Du, Changhua Meng, Yu Cheng, Weiqiang Wang, Zhouhan Lin

CASE – Condition-Aware Sentence Embeddings for Conditional Semantic Textual Similarity Measurement (2026.eacl-long)

Challenge:	Existing text embedding approaches often leverage the embeddment of the final token, typically a reserved special token such as ‘[EOS]‘.
Approach:	They propose to add a new training stage before contrastive learning to enrich the semantics of the final token embedding.
Outcome:	The proposed training stage improves performance on the Massive Text Embedding Benchmark (MTEB), achieving new state-of-the-art results across different LLM base models and scales.

Copied to clipboard

Gaifan Zhang, Yi Zhou, Danushka Bollegala

Graph Language Models (2024.acl-long)

Challenge:	Recent approaches use semantic similarity to improve the quality of sentence embeddings, but it is difficult to measure the similarity between sentences.
Approach:	They propose a condition-aware sentence embedding method that uses an LLM encoder to create an embeddable sentence under a given condition.
Outcome:	The proposed method improves the performance of LLM-based embeddings and the isotropy of the embeddable space despite requiring a small number of dimensions.

Copied to clipboard

Moritz Plenz, Anette Frank

Improving Word Embedding Factorization for Compression Using Distilled Nonlinear Neural Decomposition (2020.findings-emnlp)

Challenge:	Language Models (LMs) are the workhorses of NLP, but their interplay with structured knowledge graphs (KGs) is still actively researched.
Approach:	They propose a Graph Language Model (GLM) that integrates the strengths of both approaches and mitigates their weaknesses.
Outcome:	Empirical evaluations show that the proposed model surpasses both LM- and GNN-based baselines in supervised and zero-shot setting, demonstrating their versatility.

Copied to clipboard

Vasileios Lioutas, Ahmad Rashid, Krtin Kumar, Md. Akmal Haidar, Mehdi Rezagholizadeh

Self-Discriminative Learning for Unsupervised Document Embedding (N19-1)

Challenge:	Word-embeddings are vital components of natural language processing (NLP) but they consume a lot of memory which poses a challenge for edge deployment.
Approach:	They propose an embedding compression method based on matrix decomposition and knowledge distillation that initializes weights of pre-trained word-embeddings and fine-tunes end-to-end.
Outcome:	The proposed method has higher BLEU score on translation and lower perplexity on language modeling compared to complex, difficult to tune methods.

Copied to clipboard

Hong-You Chen, Chin-Hua Hu, Leila Wehbe, Shou-De Lin

Low-Resource NLP

Challenge:	Existing methods for document embedding learning do not consider inter-document relationships.
Approach:	They propose to exploit the inter-document information and directly model the relations of documents in embedding space with a discriminative network and a novel objective.
Outcome:	The proposed method has errors that are 5 to 13% lower than state-of-the-art models and is even more pronounced in scarce label setting.

WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models (2022.naacl-main)

Copied to clipboard

Benjamin Minixhofer, Fabian Paischer, Navid Rekabsaz

Learning multiview embeddings for assessing dementia (D18-1)

Challenge:	Existing methods to train large pretrained language models require more computational resources and are expensive to train in other languages.
Approach:	They propose a method to transfer pretrained language models to new languages using subword-based tokenization and embeddings.
Outcome:	The proposed method outperforms existing methods on low-resource languages and makes training large models more accessible and less damaging to the environment.

Copied to clipboard

Chloé Pou-Prom, Frank Rudzicz

Stock Embeddings Acquired from News Articles and Price History, and an Application to Portfolio Optimization (2020.acl-main)

Challenge:	In 2017, 5.7 million Americans were living with Alzheimer's disease (AD), and the disease accounted for $11.4 billion in healthcare costs in the United States.
Approach:	They leverage the multiview nature of a small AD dataset to learn an embedding that captures different modes of cognitive impairment.
Outcome:	The proposed embeddings achieve an F1 score of 0.82 and a mean absolute error of 3.42 in the classification task and predicting clinical scores.

Copied to clipboard

Xin Du, Kumiko Tanaka-Ishii

RelDiff: Enriching Knowledge Graph Relation Representations for Sensitivity Classification (2021.findings-emnlp)

Challenge:	Recent studies have shown that news articles can be leveraged to improve price prediction.
Approach:	They propose a method to encode the influence of news articles through a vector representation of stocks . they use a deep learning framework to acquire the vector representation using news articles and price history .
Outcome:	The proposed method can be applied to other financial problems besides price prediction.

Copied to clipboard

Hitarth Narvala, Graham McDonald, Iadh Ounis

Specializing Word Vectors by Spectral Decomposition on Heterogeneously Twisted Graphs (2020.coling-main)

Challenge:	Existing relationships between entities can be reliable indicators for classifying sensitive information, such as commercially sensitive information.
Approach:	They propose to represent entities and relations within a single embedding to better capture the relationship between the entities.
Outcome:	The proposed method significantly improves the effectiveness of sensitivity classification compared to existing methods.

Copied to clipboard

Yuanhang Ren, Ye Du

Tokenizer-Aware Cross-Lingual Adaptation of Decoder-Only LLMs through Embedding Relearning and Swapping (2026.eacl-long)

Challenge:	Word vectors have a tendency to conflate semantic similarity with semantic relatedness . a new method is proposed to retrofit word vectors with lexical constraints .
Approach:	They propose a method that heterogeneously retrofits a similarity matrix with lexical constraints.
Outcome:	The proposed method has a competitive performance compared with the state-of-the-art methods.

Copied to clipboard

Fan Jiang, Honglin Yu, Grace Y Chung, Trevor Cohn

Few-Shot Representation Learning for Out-Of-Vocabulary Words (P19-1)

Challenge:	Large Language Models (LLMs) have been primarily focused on English, leaving the multilingual ability unexplored.
Approach:	They propose a technique that creates new tokenizers and tunes embeddings on fixed model weights for target language adaptation.
Outcome:	The proposed method is light-weight and performant but has limitations for older models and high resource languages.

Copied to clipboard

Ziniu Hu, Ting Chen, Kai-Wei Chang, Yizhou Sun

ACSE: An Ancient Character Semantic-Aware Embedding for Large Language Models (2026.findings-acl)

Challenge:	Existing methods for learning word embedding assume there are enough occurrences for each word in the corpus to accurately estimate the representation of words.
Approach:	They propose to fit a representation function to predict an oracle embedding vector based on limited contexts.
Outcome:	The proposed model outperforms existing methods in constructing an accurate embedding for OOV words and improves downstream tasks when the embeddable is utilized.

Copied to clipboard

Zhihan Zhou, Daqian Shi, Lida Shi, Rui Song, Peiqiang Qiu, Xiaolei Diao, Hao Xu

EigenSent: Spectral sentence embeddings using higher-order Dynamic Mode Decomposition (P19-1)

Challenge:	Existing studies on pre-Qin documents are insufficient to understand ancient characters . ancient characters have a low level of digitization and training corpora are extremely scarce .
Approach:	They propose a semantic-aware embedding for ancient Chinese characters that integrates glyphs and lexicality into modern Chinese semantic space.
Outcome:	The proposed model integrates glyph and lexicality of ancient characters and maps them to the modern Chinese semantic space.

Copied to clipboard

Subhradeep Kayal, George Tsatsaronis

Information Extraction & Text Mining

Challenge:	Existing methods for calculating word embeddings use weighted averages or learn word representations, but this is not the case.
Approach:	They propose to use spectral methods of signal representation and summarization to construct word-sequence embeddings in unsupervised fashion by using a spectrally-based decomposition and signal summarizing technique on text.
Outcome:	The proposed method can summarize transitions in a sequence of words and generate an embedding that can represent well the sequence itself.

Analyzing the Surprising Variability in Word Embedding Stability Across Languages (2021.emnlp-main)

Copied to clipboard

Laura Burdick, Jonathan K. Kummerfeld, Rada Mihalcea

Latent Learningscape Guided In-context Learning (2024.findings-acl)

Challenge:	Word embeddings are powerful representations that form the foundation of many natural language processing architectures.
Approach:	They explore word embedding stability in a wide range of languages to gain insight into their stability.
Outcome:	The proposed results provide insights into word embedding stability in English and other languages.

Copied to clipboard

Anlai Zhou, Sunshine Jiang, Yifei Liu, Yiquan Wu, Kun Kuang, Jun Xiao

Rethinking Attribute Representation and Injection for Sentiment Classification (D19-1)

Challenge:	Existing methods to select demonstrations based on surface-level semantic similarities fall short of identifying the most fitting ones.
Approach:	They propose a method that characterizes latent learningscape features of demonstrations and uses them to create more effective prompts.
Outcome:	The proposed method outperforms leading models in arithmetic, commonsense, and symbolic reasoning tasks showing an average increase in scores by 7.4 percentage points.

Copied to clipboard

Reinald Kim Amplayo

Text Classification

Challenge:	Existing models that use text attributes to improve sentiment classification use text as a categorical feature.
Approach:	They propose to represent attributes as chunk-wise importance weight matrices and consider four locations to inject attributes.
Outcome:	The proposed method outperforms the state-of-the-art and outperformed previous models.

Embeddings for Named Entity Recognition in Geoscience Portuguese Literature (2020.lrec-1)

Copied to clipboard

Bernardo Consoli, Joaquim Santos, Diogo Gomes, Fabio Cordeiro, Renata Vieira, Viviane Moreira

imapScore: Medical Fact Evaluation Made Easy (2024.findings-acl)

Challenge:	Named Entity Recognition (NER) is a task within the field of Natural Language Processing that deals with the identification and categorization of Named entities (NEs) in a given text.
Approach:	They propose to use vector and tensor embeddings to train Portuguese Named Entity Recognition (NER) in the Geology domain.
Outcome:	The proposed model achieves state-of-the-art for the Portuguese Geology domain with one of its embeddings.

Copied to clipboard

Huimin Wang, Yutian Zhao, Xian Wu, Yefeng Zheng

Language-agnostic Representation from Multilingual Sentence Encoders for Cross-lingual Similarity Estimation (2021.emnlp-main)

Challenge:	Automated evaluation of natural language generation tasks fails to focus on medical QA because of the diversity in medical terminology.
Approach:	They propose a new data structure, imap, to capture key information in questions and answers.
Outcome:	The proposed model outperforms state-of-the-art metrics in correlation with human scores.

Copied to clipboard

Nattapong Tiyajamorn, Tomoyuki Kajiwara, Yuki Arase, Makoto Onizuka

WavRAG: Audio-Integrated Retrieval Augmented Generation for Spoken Dialogue Models (2025.acl-long)

Challenge:	Existing methods to extract language-specific information from multilingual sentence embeddings are remarkably successful in cross-lingual and multilingual NLU tasks.
Approach:	They propose to extract language-specific information from the original embedding and use it to retrieve an embeddable that fully represents the sentence’s meaning.
Outcome:	The proposed method outperforms baselines on cross-lingual sentences even in low-resource language pairs where only tens of thousands of parallel sentence pairs are available.

Copied to clipboard

Yifu Chen, Shengpeng Ji, Haoxiao Wang, Ziqing Wang, Siyu Chen, Jinzheng He, Jin Xu, Zhou Zhao

Text Embeddings Reveal (Almost) As Much As Text (2023.emnlp-main)

Challenge:	Existing RAG frameworks rely on Automatic Speech Recognition to process speech input, which discards crucial audio information and increases computational overhead.
Approach:	They propose a retrieval augmented generation framework with native, end-to-end audio support that integrates audio and text into a unified knowledge representation.
Outcome:	The proposed framework can perform 10x faster than current pipelines while delivering 10x acceleration.

Copied to clipboard

John Morris, Volodymyr Kuleshov, Vitaly Shmatikov, Alexander Rush

Relation-aware Ensemble Learning for Knowledge Graph Embedding (2023.emnlp-main)

Challenge:	a vector database of dense text embeddings stores only the text data, not the original text . a multi-step method that iteratively corrects and re-embeds text can recover 92% of 32-token text inputs exactly.
Approach:	They propose a method that iteratively corrects and re-embeds text to recover 92% of 32-token text inputs exactly.
Outcome:	The proposed method recovers 92% of 32-token text inputs exactly.

Copied to clipboard

Ling Yue, Yongqi Zhang, Quanming Yao, Yong Li, Xian Wu, Ziheng Zhang, Zhenxi Lin, Yefeng Zheng

Concept Tokens: Learning Behavioral Embeddings Through Concept Definitions (2026.findings-acl)

Challenge:	Existing methods to explore semantics of knowledge graphs have been proposed to explore these semantics in distinct ways.
Approach:	They propose to leverage existing methods in relation-aware manner to learn an ensemble by leveraging existing methods.
Outcome:	The proposed method has the same computation cost as general ensemble methods but with much better performance on benchmark datasets.

Copied to clipboard

Ignacio Sastre, Aiala Rosá

Probing Relative Interaction and Dynamic Calibration in Multi-modal Entity Alignment (2025.acl-long)

Challenge:	Concept Tokens is a lightweight method that adds a special token to a pretrained LLM . we find that negating the hallucination token reduces hallucines and lowers precision .
Approach:	They propose a lightweight method that adds a new special token to a pretrained LLM and learns only its embedding from multiple natural language definitions of a target concept.
Outcome:	The proposed method can learn only its embedding from multiple definitions of a target concept . the study shows that it can improve hallucinations and recasting in closed-book questions .

Copied to clipboard

Chenxiao Li, Jingwei Cheng, Qiang Tong, Fu Zhang, Cairui Wang

Embedding-based In-Context Prompt Training for Enhancing LLMs as Text Encoders (2026.findings-acl)

Challenge:	Current methods for multi-modal entity alignment ignore relative interactions between modalities and the accuracy of weights.
Approach:	They propose a relative interaction and calibration framework for multi-modal entity alignment that uses attention mechanisms to perceive the uncertainty of the weight for each modality.
Outcome:	The proposed framework outperforms baselines across 5 datasets and 23 settings.

Copied to clipboard

Ailiang Lin, Zhuoyun Li, Keyu Mao, Kotaro Funakoshi, Manabu Okumura