| Challenge: | a novel approach to document retrieval can be used to encode documents as vectors . a few query-relevant terms can be pruned out to reduce index overhead . |
| Approach: | They propose to enhance the document representation for the bi-encoder approach in dense document retrieval. |
| Outcome: | The proposed solution reduces latency and memory footprint up to 8- and 3-fold . it is validated on MSMARCO and real-world search query logs . |
Similar Papers
Improving Document Representations by Generating Pseudo Query Embeddings for Dense Retrieval (2021.acl-long)
Copied to clipboard
| Challenge: | Existing retrieval models based on dense representations show better performance than sparse representations. |
| Approach: | They propose a method to mimic the queries to each of the documents by an iterative clustering process and represent the documents using multiple pseudo queries. |
| Outcome: | The proposed model achieves state-of-the-art results on a large dataset while remaining high efficiency. |
Investigating Multi-layer Representations for Dense Passage Retrieval (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Dense retrieval models adopt vectors from the last hidden layer of the document encoder to represent a document, which is in contrast to the fact that representations in different layers of a pre-trained language model contain different kinds of linguistic knowledge and behave differently during fine-tuning. |
| Approach: | They propose to utilize representations from multiple encoder layers to make up the representation of a document, which they denote Multi-layer Representations (MLR). |
| Outcome: | The proposed model outperforms dual encoder, ME-BERT and ColBERT in the single-vector retrieval setting and with other advanced training techniques. |
Sparse, Dense, and Attentional Representations for Text Retrieval (2021.tacl-1)
Copied to clipboard
| Challenge: | Dual encoders perform retrieval by encoding documents and queries into dense low-dimensional vectors, scoring each document by its inner product with the query. |
| Approach: | They propose a dual-encoder-based neural model that combines the efficiency of dual encoders with expressiveness of more costly attentional architectures. |
| Outcome: | The proposed model outperforms strong alternatives in large-scale retrieval. |
Augmenting Document Representations for Dense Retrieval with Interpolation and Perturbation (2022.acl-short)
Copied to clipboard
| Challenge: | Existing sparse retrieval models rely on term-based matching to retrieve relevant documents. |
| Approach: | They propose a framework which augments the representations of documents with interpolation and perturbation. |
| Outcome: | The proposed framework significantly outperforms baselines on the dense retrieval of both the labeled and unlabeled documents. |
Generative Multi-hop Retrieval (2022.emnlp-main)
Copied to clipboard
| Challenge: | A bi-encoder approach to text retrieval has limitations in multi-hop settings; the reformulated query gets longer as the number of hops increases, which further tightens the embedding bottleneck of the query vector. |
| Approach: | They propose an encoder-decoder model that performs multi-hop retrieval by simply generating the entire text sequences of the retrieval targets. |
| Outcome: | The proposed model achieves comparable or higher performance than bi-encoder models in five datasets while demonstrating superior GPU memory and storage footprint. |
Extremely efficient online query encoding for dense retrieval (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing dense retrieval systems use the same model architecture for encoding both passages and queries, even though queries are much shorter and simpler than passages. |
| Approach: | They propose a small efficient RNN query encoder that can reduce latency by 12 with only a minor decrease in quality. |
| Outcome: | The proposed solution reduces latency by up to 12 while achieving 35.5 MRR@10 score. |
Multi-View Document Representation Learning for Open-Domain Dense Retrieval (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for dense retrieval are hard to match with multiple views. |
| Approach: | They propose a multi-view document representation learning framework to generate multiple embeddings through viewers to represent documents and enforce them to align with different queries. |
| Outcome: | The proposed method outperforms recent works and achieves state-of-the-art results. |
CAPSTONE: Curriculum Sampling for Dense Retrieval with Document Expansion (2023.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that dense retrieval models are better at obtaining query-informed representations. |
| Approach: | They propose a dual-encoder approach that computes latent representations of query and document independently, but inference replaces the real query with a generated one. |
| Outcome: | The proposed approach outperforms previous dense retrieval models on in-domain and out-of-domain datasets. |
XTR meets ColBERTv2: Adding ColBERTv2 Optimizations to XTR (2025.coling-industry)
Copied to clipboard
| Challenge: | XTR eliminates the need for multi-stage retrieval, but doesn't incorporate efficiency optimizations from ColBERTv2 which improve indexing and retrieval speed. |
| Approach: | They propose a multi-vector retrieval method that simplifies retrieval into a single stage through a modified learning objective. |
| Outcome: | The proposed method eliminates the need for multistage retrieval but doesn't incorporate efficiency optimizations from ColBERTv2 which improve indexing and retrieval speed. |
Leveraging Cognitive Complexity of Texts for Contextualization in Dense Retrieval (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to estimate semantic similarity of queries and documents rely on token-level information derived from query/document interactions. |
| Approach: | They propose a new DRM that leverages query/document interactions based on full embedding representations generated by a Transformer-based model. |
| Outcome: | The proposed model outperforms fine-tuning techniques on lightweight bi-encoders and traditional late-interaction models. |