Drift-Adapter: A Practical Approach to Near Zero-Downtime Embedding Model Upgrades in Vector Databases (2025.emnlp-main)
Copied to clipboard
| Challenge: | Upgrading embedding models in production environments requires re-encoding the entire corpus and rebuilding the Approximate Nearest Neighbor (ANN) index. |
| Approach: | They propose a lightweight, learnable transformation layer designed to bridge embedding spaces between models by mapping new queries into the legacy embeddable space. |
| Outcome: | The proposed transformation layer recovers 95–99% of the retrieval recall of a full re-embedding, adding less than 10,s query latency. |
Similar Papers
Embedding-Converter: A Unified Framework for Cross-Model Embedding Transformation (2025.acl-long)
Copied to clipboard
| Challenge: | Embedding models are fundamental to modern machine learning, but the continuous development of new models presents a major challenge. |
| Approach: | They propose a framework for efficiently transforming embeddings between different models, thus avoiding costly ‘re-embedding’. |
| Outcome: | The proposed framework achieves 100 times faster and cheaper computations in real-world applications. |
Using Context-to-Vector with Graph Retrofitting to Improve Word Embeddings (2022.acl-long)
Copied to clipboard
| Challenge: | Contextualized embeddings are expensive and resource-demanding, hence environmentally unfriendly. |
| Approach: | They propose a method to convert contextualized embeddings from pre-trained models into static embeddables using synonym knowledge and weighted vector distribution. |
| Outcome: | The proposed method outperforms baseline embeddings by a large margin through extrinsic and intrinsic tasks. |
AdapterHub: A Framework for Adapting Transformers (2020.emnlp-demos)
Copied to clipboard
Jonas Pfeiffer, Andreas Rücklé, Clifton Poth, Aishwarya Kamath, Ivan Vulić, Sebastian Ruder, Kyunghyun Cho, Iryna Gurevych
| Challenge: | AdapterHub framework enables dynamic “stiching-in” of pre-trained adapters for different tasks and languages. |
| Approach: | They propose a framework that allows dynamic "stiching-in" of pre-trained adapters for different tasks and languages. |
| Outcome: | The proposed framework allows dynamic “stiching-in” of pre-trained adapters for different tasks and languages. |
Search-Adaptor: Embedding Customization for Information Retrieval (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods to embed text in large language models are limited to zero-shot setups and can be integrated with any LLM. |
| Approach: | They propose a method for customizing LLMs for information retrieval by modifying the embeddings generated by pre-trained LLM models and can be integrated with any LLM. |
| Outcome: | The proposed method improves performance on English, multilingual, and multimodal retrieval datasets by 5% over 14 BEIR datasets. |
Mini-Model Adaptation: Efficiently Extending Pretrained Models to New Languages via Aligned Shallow Training (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to pretrain Masked Language Models (MLMs) are expensive and require a full forward and backward pass over the entire model. |
| Approach: | They propose to learn a shallow mini-model from a fraction of a large model's parameters and plug it into a larger model for rapid cross-lingual transfer. |
| Outcome: | Experiments on XNLI, MLQA and PAWS-X show that mini-model adaptation matches the standard approach using up to 2.3x less compute on average. |
Tokenizer-Aware Cross-Lingual Adaptation of Decoder-Only LLMs through Embedding Relearning and Swapping (2026.eacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been primarily focused on English, leaving the multilingual ability unexplored. |
| Approach: | They propose a technique that creates new tokenizers and tunes embeddings on fixed model weights for target language adaptation. |
| Outcome: | The proposed method is light-weight and performant but has limitations for older models and high resource languages. |
Matryoshka-Adaptor: Unsupervised and Supervised Tuning for Smaller Embedding Dimensions (2024.emnlp-main)
Copied to clipboard
| Challenge: | Embeddings from Large Language Models (LLMs) have emerged as critical components in information retrieval applications. |
| Approach: | They propose a tuning framework for the customization of LLM embeddings. |
| Outcome: | The proposed framework reduces embedding dimensions while maintaining comparable performance levels. |
ELLA: Efficient Lifelong Learning for Adapters in Large Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing approaches to training Large Language Models (LLMs) suffer from catastrophic forgetting when adapted sequentially to new tasks in a continual learning (CL) setting. Existing methods are impractical and could potentially violate privacy. |
| Approach: | They propose a training framework built on the principle of selective subspace de-correlation that characterizes the structure of past updates and penalizes alignments along their high-energy, task-specific directions. |
| Outcome: | The proposed training framework achieves state-of-the-art CL performance on three popular benchmarks spanning both classification and generative tasks with relative accuracy gains of up to 9.6% and a 35 smaller memory footprint. |
Adapters: A Unified Library for Parameter-Efficient and Modular Transfer Learning (2023.emnlp-demo)
Copied to clipboard
Clifton Poth, Hannah Sterz, Indraneil Paul, Sukannya Purkayastha, Leon Engländer, Timo Imhof, Ivan Vulić, Sebastian Ruder, Iryna Gurevych, Jonas Pfeiffer
| Challenge: | Adapters is an open-source library that unifies parameter-efficient and modular transfer learning in large language models. |
| Approach: | They propose to integrate 10 different methods into a unified interface for parameter-efficient and modular transfer learning in large language models. |
| Outcome: | The proposed library is able to perform on multiple NLP tasks and is open-source. |
Advancing Vision-Language Models with Adapter Ensemble Strategies (2024.findings-emnlp)
Copied to clipboard
| Challenge: | CLIP revolutes vision-language pretraining by using contrastive learning on paired web data. |
| Approach: | They propose to combine a "adapter ensemble" with traditional machine learning techniques to augment large-scale pretrained vision-language models. |
| Outcome: | The proposed model outperforms baselines and derives improvement when the number of ensemble parameters increases. |