| Challenge: | Current methods based on contrastive learning have generated high-quality sentence embeddings. |
| Approach: | They propose a method to enhance LLM performance on sentence embeddings with a one-word limitation. |
| Outcome: | The proposed method outperforms contrastive learning methods on sentence embeddings without fine-tuning and with fine-untun. |
Similar Papers
Meta-Task Prompting Elicits Embeddings from Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for large language modeling are based on task-related instructions or prompts. |
| Approach: | They propose a method for generating high-quality sentence embeddings from Large Language Models (LLMs) using meta-task prompts. |
| Outcome: | The proposed method produces high-quality sentences without fine-tuning . it excels on STS benchmarks and in downstream tasks, surpassing models with similar prompts . |
Improving Sentence Embeddings with Automatic Generation of Training Data Using Few-shot Examples (2024.acl-srw)
Copied to clipboard
| Challenge: | Decoder-based large language models (LLMs) have shown high performance on many tasks in natural language processing. |
| Approach: | They propose to automatically generate an NLI dataset with an LLM and use it for fine-tuning of PromptEOL. |
| Outcome: | The proposed model outperforms existing models on STS tasks without large manually annotated datasets. |
SemPA: Improving Sentence Embeddings of Large Language Models through Semantic Preference Alignment (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing sentence embedding methods rely on fixed prompt templates or involve modifications to the model architecture, compromising its generative capabilities. |
| Approach: | They propose a sentence-level direct preference optimization approach that boosts the sentence representations while preserving the generative ability of LLMs. |
| Outcome: | The proposed method improves representations of semantically meaningful vectors without sacrificing generation capability. |
Embedding-based In-Context Prompt Training for Enhancing LLMs as Text Encoders (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been widely explored for embedding generation. |
| Approach: | They propose an embedding-based in-context prompt training strategy that leverages in-constext learning to generate high-quality embeddables while reducing computational burden. |
| Outcome: | The proposed method surpasses models trained on publicly available retrieval data and achieves state-of-the-art embedding performance on the MTEB benchmark. |
Refining Sentence Embedding Model through Ranking Sentences Generation with Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Sentence embedding is essential for many NLP tasks, but reliance on manual labels limits scalability. |
| Approach: | They propose a method for controlling the generation direction of large language models in the latent space by integrating ranking information and semantic information. |
| Outcome: | The proposed method achieves new SOTA performance with a modest cost in ranking sentence synthesis. |
Improving Text Embeddings with Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for obtaining text embeddings require complex training pipelines . authors leverage proprietary LLMs to generate diverse synthetic data for text embeds based on 93 languages . |
| Approach: | They propose a method for obtaining high-quality text embeddings using only synthetic data and less than 1k training steps. |
| Outcome: | The proposed method achieves strong performance on competitive text embedding benchmarks without using any labeled data. |
Bridging the Language Gap: Dynamic Learning Strategies for Improving Multilingual Performance in LLMs (2025.coling-main)
Copied to clipboard
Somnath Kumar, Vaibhav Balloli, Mercy Ranjit, Kabir Ahuja, Sunayana Sitaram, Kalika Bali, Tanuja Ganu, Akshay Nambi
| Challenge: | Large language models (LLMs) excel in diverse applications but still struggle with non-Latin scripts and low-resource languages. |
| Approach: | They propose a dynamic learning approach that optimizes prompt strategy, embedding model, and LLM per query at runtime. |
| Outcome: | The proposed approach achieves 10-15% improvements in multilingual performance over pre-trained models and 4x gains compared to fine-tuned, language-specific models. |
ULLME: A Unified Framework for Large Language Model Embeddings with Generation-Augmented Learning (2024.emnlp-demo)
Copied to clipboard
| Challenge: | Existing frameworks for large language model embeddings have limited support for only a limited range of architectures and fine-tuning strategies. |
| Approach: | They propose a framework that enables bidirectional attention across various LLMs and supports a range of fine-tuning strategies. |
| Outcome: | The proposed framework enables bidirectional attention across various LLMs and supports a range of fine-tuning strategies. |
Small Models are Valuable Plug-ins for Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large-scale pre-trained language models are difficult to fine-tune due to their huge weights and limited context length. |
| Approach: | They propose an approach which allows black-box LLMs to work with locally fine-tuned smaller models, resulting in superior performance on supervised tasks. |
| Outcome: | The proposed approach overcomes the challenges of poor performance and instability of In-Context Learning (ICL) while reducing the complexity of in-context learning. |
Generating Datasets with Pretrained Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent approaches to obtain high-quality sentence embeddings from pretrained language models require labeled data or finetuned on large set of labeles. |
| Approach: | They propose to use generative abilities of large and high-performing PLMs to generate entire datasets of labeled text pairs from scratch and fine tune much smaller and more efficient models. |
| Outcome: | The proposed approach outperforms baselines on several semantic textual similarity datasets. |