Papers with MS-MARCO
Generating a Common Question from Multiple Documents using Multi-source Encoder-Decoder Models (D19-56)
Copied to clipboard
| Challenge: | Ambiguous user queries can result in multiple topics being retrieved from search engines. |
| Approach: | They propose a task of generating a common question from multiple documents by training an RNN-based single encoder-decoder generator from document pairs and then a model that aggregates these word distributions to generate a question. |
| Outcome: | The proposed model significantly outperforms existing models when evaluated using automated metrics and human judgments on the MS-MARCO-QA dataset. |
Dwell in the Beginning: How Language Models Embed Long Documents for Dense Retrieval (2024.acl-short)
Copied to clipboard
| Challenge: | Existing studies have shown that Transformer-based language models lose information in the middle of input sequences, especially in the context of web document retrieval. |
| Approach: | They examine position biases at multiple stages of the training pipeline for an encoder-decoder neural retrieval model, namely language model pre-training, contrastive pre- training, and contrastive fine-tuning. |
| Outcome: | The proposed model generates embeddings that better capture the beginning of the input content, with fine-tuning further aggravating this effect. |
D3: Dynamic Docid Decoding for Multi-Intent Generative Retrieval (2026.eacl-industry)
Copied to clipboard
| Challenge: | Existing GR systems rely on offline DocID assignment and constrained decoding . offline Doc ID assignment and decoding often prevents GR from capturing query-specific intent . |
| Approach: | They propose a mechanism that adaptively refines DocIDs through query-informed identifier expansion. |
| Outcome: | The proposed mechanism improves retrieval accuracy on unseen and multi-intent documents. |
DPTDR: Deep Prompt Tuning for Dense Passage Retrieval (2022.coling-1)
Copied to clipboard
| Challenge: | Recent studies show that prompt tuning is unfriendly for industrial deployment in dense retrieval tasks. |
| Approach: | They propose to apply prompt tuning to dense retrieval tasks to reduce deployment cost . they propose to use retrieval-oriented intermediate pretraining and unified negative mining . |
| Outcome: | The proposed method outperforms state-of-the-art models on MS-MARCO and Natural Questions. |
Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval (2022.acl-long)
Copied to clipboard
| Challenge: | Recent research shows that fine-tuning dense retrievers to realize their capacity requires carefully designed fine-cuning techniques. |
| Approach: | They propose a pre-training architecture that learns to condense information into the dense vector through LM pre-training and a coCondenser architecture which adds an unsupervised corpus-level contrastive loss to warm up the passage embedding space. |
| Outcome: | The proposed architecture reduces the need for heavy data engineering and large batch training. |
DUQGen: Effective Unsupervised Domain Adaptation of Neural Rankers by Diversifying Synthetic Query Generation (2024.naacl-long)
Copied to clipboard
| Challenge: | State-of-the-art rankers pre-trained on large task-specific training data such as MS-MARCO exhibit strong performance on various ranking tasks without domain adaptation, also called zero-shot. |
| Approach: | They propose a method to generate unsupervised domain adaptation for ranking using large-scale task-specific training data such as MS-MARCO and Wikipedia retrieval. |
| Outcome: | The proposed method outperforms all zero-shot baselines and significantly outperfies the SOTA baselines on 16 out of 18 datasets, for an average of 4% relative improvement across all datasets. |
Multi-Vector Attention Models for Deep Re-ranking (2021.emnlp-main)
Copied to clipboard
| Challenge: | Document retrieval systems often use two styles of neural network models . dual encoder models are used for retrieval and deep re-ranking, while cross-attention models are typically used for shallow reranking. |
| Approach: | They propose a dual encoder and cross-attention neural network architectures that combine query and document representations to optimize retrieval accuracy. |
| Outcome: | The proposed architecture trades off retrieval accuracy with joint computation and offline document storage cost. |
Query2doc: Query Expansion with Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for sparse and dense retrieval have limited success on popular datasets. |
| Approach: | They propose a query expansion approach that generates pseudo-documents by few-shot prompting large language models and then expands the query with generated pseudo-docs. |
| Outcome: | The proposed method boosts the performance of BM25 on ad-hoc IR datasets by 3% to 15% without any model fine-tuning. |
Noisy Self-Training with Synthetic Queries for Dense Retrieval (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing neural retrieval models require training on a sufficient number of human-labelled query-passage pairs to work well. |
| Approach: | They propose a noisy self-training framework with synthetic queries to improve retrieval methods. |
| Outcome: | The proposed method outperforms baselines on general-domain and out-of-domain retrieval benchmarks on low-resource settings and is data efficient and data efficient. |
RADCoT: Retrieval-Augmented Distillation to Specialization Models for Generating Chain-of-Thoughts in Query Expansion (2024.lrec-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated superior performance to that of small language models in information retrieval for various subtasks including dense retrieval, reranking, query expansion, and pseudo-document generation. |
| Approach: | They propose a retrieval-augmented model specialization that distills the capability of LLMs to generate the chain-of-thoughts (CoT) for query expansion into a RADCoT. |
| Outcome: | The proposed model can generate the chain-of-thoughts (CoT) for query expansion, reducing the burden of internalizing and retaining world knowledge in model parameters. |