CAPSTONE: Curriculum Sampling for Dense Retrieval with Document Expansion (2023.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that dense retrieval models are better at obtaining query-informed representations. |
| Approach: | They propose a dual-encoder approach that computes latent representations of query and document independently, but inference replaces the real query with a generated one. |
| Outcome: | The proposed approach outperforms previous dense retrieval models on in-domain and out-of-domain datasets. |
Similar Papers
ExpandR: Teaching Dense Retrievers Beyond Queries with LLM Guidance (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for enhancing dense retrieval with query augmentation ignore the alignment between generation and ranking objectives. |
| Approach: | They propose a unified LLM-augmented dense retrieval framework that jointly optimizes both the LLM and the retriever. |
| Outcome: | Experimental results show that ExpandR outperforms strong baselines, achieving more than 5% improvement in retrieval performance. |
Empowering Dual-Encoder with Query Generator for Cross-Lingual Dense Retrieval (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to distill knowledge from cross-encoder re-ranker to dual-encoding retriever are lacking in the cross-lingual setting. |
| Approach: | They propose to use a query generator as the teacher in the cross-lingual setting to distill knowledge to a dual-encoder retrieval model. |
| Outcome: | The proposed method outperforms state-of-the-art methods on two benchmark datasets. |
Improving Document Representations by Generating Pseudo Query Embeddings for Dense Retrieval (2021.acl-long)
Copied to clipboard
| Challenge: | Existing retrieval models based on dense representations show better performance than sparse representations. |
| Approach: | They propose a method to mimic the queries to each of the documents by an iterative clustering process and represent the documents using multiple pseudo queries. |
| Outcome: | The proposed model achieves state-of-the-art results on a large dataset while remaining high efficiency. |
Pseudo-Relevance for Enhancing Document Representation (2022.emnlp-main)
Copied to clipboard
| Challenge: | a novel approach to document retrieval can be used to encode documents as vectors . a few query-relevant terms can be pruned out to reduce index overhead . |
| Approach: | They propose to enhance the document representation for the bi-encoder approach in dense document retrieval. |
| Outcome: | The proposed solution reduces latency and memory footprint up to 8- and 3-fold . it is validated on MSMARCO and real-world search query logs . |
GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval (2022.naacl-main)
Copied to clipboard
| Challenge: | Dense retrieval approaches suffer from the lexical gap and require large amounts of training data. |
| Approach: | They propose an unsupervised method for domain adaptation that uses query generator and pseudo labeling from a cross-encoder to improve retrieval performance. |
| Outcome: | The proposed method outperforms state-of-the-art retrieval methods on domain-specialized datasets by 9.3 points nDCG@10 on six tasks. |
Sparse, Dense, and Attentional Representations for Text Retrieval (2021.tacl-1)
Copied to clipboard
| Challenge: | Dual encoders perform retrieval by encoding documents and queries into dense low-dimensional vectors, scoring each document by its inner product with the query. |
| Approach: | They propose a dual-encoder-based neural model that combines the efficiency of dual encoders with expressiveness of more costly attentional architectures. |
| Outcome: | The proposed model outperforms strong alternatives in large-scale retrieval. |
Large Dual Encoders Are Generalizable Retrievers (2022.emnlp-main)
Copied to clipboard
Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gustavo Hernandez Abrego, Ji Ma, Vincent Zhao, Yi Luan, Keith Hall, Ming-Wei Chang, Yinfei Yang
| Challenge: | Experimental results show that dual encoders outperform sparse and dense retrievers on the BEIR dataset significantly. |
| Approach: | They challenge belief that bottleneck layer is too limited for out-of-domain generalization . they scale up the model while keeping bottleneck as a single dot-product with a fixed size . |
| Outcome: | The proposed model outperforms sparse and dense retrievers on the BEIR dataset significantly. |
Augmenting Document Representations for Dense Retrieval with Interpolation and Perturbation (2022.acl-short)
Copied to clipboard
| Challenge: | Existing sparse retrieval models rely on term-based matching to retrieve relevant documents. |
| Approach: | They propose a framework which augments the representations of documents with interpolation and perturbation. |
| Outcome: | The proposed framework significantly outperforms baselines on the dense retrieval of both the labeled and unlabeled documents. |
Query2doc: Query Expansion with Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for sparse and dense retrieval have limited success on popular datasets. |
| Approach: | They propose a query expansion approach that generates pseudo-documents by few-shot prompting large language models and then expands the query with generated pseudo-docs. |
| Outcome: | The proposed method boosts the performance of BM25 on ad-hoc IR datasets by 3% to 15% without any model fine-tuning. |
Query-as-context Pre-training for Dense Passage Retrieval (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to improve passage retrieval performance by using context-supervised pre-training are weakly correlated. |
| Approach: | They propose to use query-as-context pre-training to train passage-query pairs . they evaluate the pre-trained models on large-scale passage retrieval benchmarks . |
| Outcome: | The proposed technique improves performance on large-scale passage retrieval benchmarks and out-of-domain zero-shot benchmarks. |