Meta-Task Prompting Elicits Embeddings from Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for large language modeling are based on task-related instructions or prompts. |
| Approach: | They propose a method for generating high-quality sentence embeddings from Large Language Models (LLMs) using meta-task prompts. |
| Outcome: | The proposed method produces high-quality sentences without fine-tuning . it excels on STS benchmarks and in downstream tasks, surpassing models with similar prompts . |
Similar Papers
Scaling Sentence Embeddings with Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Current methods based on contrastive learning have generated high-quality sentence embeddings. |
| Approach: | They propose a method to enhance LLM performance on sentence embeddings with a one-word limitation. |
| Outcome: | The proposed method outperforms contrastive learning methods on sentence embeddings without fine-tuning and with fine-untun. |
Towards Unified Task Embeddings Across Multiple Models: Bridging the Gap for Prompt-Based Large Language Models and Beyond (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing task embedding methods rely on fine-tuned, task-specific language models, which hinders their adaptability to prompt-guided Large Language Models (LLMs). |
| Approach: | They propose a framework for unified task embedding that harmonizes task embeds from various models within a single vector space. |
| Outcome: | The proposed framework harmonizes task embeddings from various models within a single vector space. |
Contrastive Prompting Enhances Sentence Embeddings in LLMs through Inference-Time Steering (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies focus on prompt engineering to encode the full semantics of a sentence into the embedding of the last token. |
| Approach: | They propose a technique that introduces an extra auxiliary prompt to elicit better sentence embedding . they propose to use the hidden state of the token as the sentence embedded in LLMs . |
| Outcome: | The proposed technique can improve performance of existing prompt-based methods on STS tasks and downstream classification tasks. |
Embedding-based In-Context Prompt Training for Enhancing LLMs as Text Encoders (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been widely explored for embedding generation. |
| Approach: | They propose an embedding-based in-context prompt training strategy that leverages in-constext learning to generate high-quality embeddables while reducing computational burden. |
| Outcome: | The proposed method surpasses models trained on publicly available retrieval data and achieves state-of-the-art embedding performance on the MTEB benchmark. |
Improving Text Embeddings with Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for obtaining text embeddings require complex training pipelines . authors leverage proprietary LLMs to generate diverse synthetic data for text embeds based on 93 languages . |
| Approach: | They propose a method for obtaining high-quality text embeddings using only synthetic data and less than 1k training steps. |
| Outcome: | The proposed method achieves strong performance on competitive text embedding benchmarks without using any labeled data. |
Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Prompts (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are known to perform tasks by simply observing few exemplars, but performance among under-represented languages falls behind due to pre-training data imbalance. |
| Approach: | They propose to assemble synthetic exemplars from high-resource languages to prompt LLMs to translate from any language into English and use them to create intra-lingual exemplar models to perform tasks in target languages. |
| Outcome: | The proposed method outperforms supervised few-shot learning in LLMs of different sizes for translations between English and 13 Indic and 21 African low-resource languages. |
Beyond Prompting: An Efficient Embedding Framework for Open-Domain Question Answering (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have recently pushed open-domain question answering (ODQA) to new heights. |
| Approach: | They propose an embedding-level framework that enhances both the retriever and the reader by reordering query representations via lightweight linear layers under an unsupervised contrastive learning objective. |
| Outcome: | The proposed framework outperforms baselines in accuracy and efficiency across three open-source LLMs, three retrieval methods, and four ODQA benchmarks. |
Task-oriented Domain-specific Meta-Embedding for Text Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods neglect domain-specific knowledge and use the same word embedding for each word in all domain-specified datasets. |
| Approach: | They propose a method to incorporate domain-specific and task-oriented information into meta-embeddings by combining pre-trained word embeddings. |
| Outcome: | The proposed method performs well on four text classification datasets and shows that it is compatible with existing methods. |
PromptReps: Prompting Large Language Models to Generate Dense and Sparse Representations for Zero-Shot Document Retrieval (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) excel in zero-shot document ranking tasks. |
| Approach: | They propose a prompt-based re-ranking method that requires no further training but is only feasible for reranking a handful of candidates due to computational costs. |
| Outcome: | The proposed method can retrieve documents from the entire corpus without training and with a large amount of paired text data. |
LLM-Rec: Personalized Recommendation via Prompting Large Language Models (2024.findings-naacl)
Copied to clipboard
Hanjia Lyu, Song Jiang, Hanqing Zeng, Yinglong Xia, Qifan Wang, Si Zhang, Ren Chen, Chris Leung, Jiajie Tang, Jiebo Luo
| Challenge: | Recent advances in large language models (LLMs) have showcased their remarkable ability to harness commonsense knowledge and reasoning. |
| Approach: | They propose a novel approach which incorporates four distinct prompting strategies of text enrichment for improving personalized text-based recommendations. |
| Outcome: | The proposed approach improves recommendation quality and even basic MLP models achieve comparable or even better results than complex content-based methods. |