Papers by Ailiang Lin
Causal2Vec: Improving Decoder-only LLMs as Embedding Models through a Contextual Token (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods modify attention mechanism to be bidirectional, undermining LLMs’ ability to extract semantic information acquired during pre-training. |
| Approach: | They propose a general-purpose embedding model that pre-encodes input text into a single Contextual token and then prepends it to the LLM's input sequence. |
| Outcome: | The proposed model improves performance of decoder-only large language models without altering their architectures or introducing significant computational overhead. |
Embedding-based In-Context Prompt Training for Enhancing LLMs as Text Encoders (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been widely explored for embedding generation. |
| Approach: | They propose an embedding-based in-context prompt training strategy that leverages in-constext learning to generate high-quality embeddables while reducing computational burden. |
| Outcome: | The proposed method surpasses models trained on publicly available retrieval data and achieves state-of-the-art embedding performance on the MTEB benchmark. |