Relevance-assisted Generation for Robust Zero-shot Retrieval (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Despite strong in-domain performance, dense retrievers have shown poor generalization to out-of-domain zero-shot tasks where no training queries are available. |
| Approach: | They propose to generate domain-specific pseudo queries for fine-tuning with domain-relevant relevance between PQ and documents. |
| Outcome: | The proposed approach is more robust to domain shifts, validated on BEIR zero-shot tasks. |
Similar Papers
Zero-shot Neural Passage Retrieval via Domain-targeted Synthetic Question Generation (2021.eacl-main)
Copied to clipboard
| Challenge: | Recent advances in neural retrieval have led to advancements on document, passage and knowledge-base benchmarks. |
| Approach: | They propose an approach to zero-shot learning for passage retrieval that uses synthetic question generation to close this gap. |
| Outcome: | The proposed approach can exceed term-based techniques on document retrieval benchmarks by using domain-targeted synthetic question generation. |
It’s All Relative! – A Synthetic Query Generation Approach for Improving Zero-Shot Relevance Prediction (2024.findings-naacl)
Copied to clipboard
| Challenge: | Large language models generate synthetic query-document pairs by prompting with as few as 8 demonstrations. |
| Approach: | They propose to generate queries simultaneously for different labels by prompting with 8 demonstrations. |
| Outcome: | Extensive experimentation shows that synthetic queries generated in such a fashion improve performance. |
Zero-Shot Dense Retrieval with Momentum Adversarial Domain Invariant Representations (2022.findings-acl)
Copied to clipboard
| Challenge: | Dense retrieval (DR) methods first encode texts into a dense embedding space and then conduct text retrieval using efficient nearest neighbor search. |
| Approach: | They propose Momentum adversarial Domain Invariant Representation learning to train a domain classifier that distinguishes source versus target domains and adversarially updates the DR encoder to learn domain invariant representations. |
| Outcome: | The proposed method outperforms baselines on 10+ ranking datasets collected in the BEIR benchmark in the zero-shot setting, with more than 10% relative gains on datasets with enough sensitivity for DR models’ evaluation. |
DUQGen: Effective Unsupervised Domain Adaptation of Neural Rankers by Diversifying Synthetic Query Generation (2024.naacl-long)
Copied to clipboard
| Challenge: | State-of-the-art rankers pre-trained on large task-specific training data such as MS-MARCO exhibit strong performance on various ranking tasks without domain adaptation, also called zero-shot. |
| Approach: | They propose a method to generate unsupervised domain adaptation for ranking using large-scale task-specific training data such as MS-MARCO and Wikipedia retrieval. |
| Outcome: | The proposed method outperforms all zero-shot baselines and significantly outperfies the SOTA baselines on 16 out of 18 datasets, for an average of 4% relative improvement across all datasets. |
Domain Adaptation for Dense Retrieval and Conversational Dense Retrieval through Self-Supervision by Meticulous Pseudo-Relevance Labeling (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent studies have shown that dense retrieval models generalize less well than interaction-based models on out-of-distribution data sets. |
| Approach: | They propose to combine query-generation approach with self-supervision approach in which pseudo-relevance labels are automatically generated on the target domain. |
| Outcome: | The proposed approach is based on a T5-3B model for pseudo-positive labeling and hard negatives on conversational dense retrieval models. |
Towards Robust Neural Retrieval with Source Domain Synthetic Pre-Finetuning (2022.coling-1)
Copied to clipboard
Revanth Gangi Reddy, Vikas Yadav, Md Arafat Sultan, Martin Franz, Vittorio Castelli, Heng Ji, Avirup Sil
| Challenge: | Existing neural IR systems rely on lexical matching for query-passage alignment, while masked language models use a dual encoder architecture to encode passages and questions into continuous vector representations. |
| Approach: | They propose to enhance the out-of-domain generalization of Dense Passage Retrieval (DPR) through synthetic data augmentation only in the source domain. |
| Outcome: | The proposed model outperforms existing models in in-domain and zero-shot evaluations on Wikipedia-based datasets. |
Precise Zero-Shot Dense Retrieval without Relevance Labels (2023.acl-long)
Copied to clipboard
| Challenge: | Existing dense retrieval systems that use semantic embedding similarities can be effective across tasks and languages. |
| Approach: | They propose to pivot through Hypothetical Document Embeddings (HyDE) given a query, HyDE first zero-shot prompts an instruction-following language model to generate a hypothetical document. |
| Outcome: | The proposed method significantly outperforms the state-of-the-art unsupervised dense retriever Contriever and shows strong performance comparable to fine-tuned retrievers across tasks and languages. |
MixGR: Enhancing Retriever Generalization for Scientific Domain through Complementary Granularity (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show the importance of document retrieval in the scientific domain. |
| Approach: | They propose a zero-shot approach to measure query-document similarity using atomic components in queries and documents to combine them into a united score. |
| Outcome: | The proposed approach outperforms previous document retrieval methods by 24.7%, 9.8%, and 6.9% on nDCG@5 with unsupervised, supervised, and LLM-based retrievers. |
Disentangling Questions from Query Generation for Task-Adaptive Retrieval (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work generates synthetic queries from domain-specific documents to jointly train the retriever. |
| Approach: | They propose a query generator that better adapts to wide search intents expressed in the BeIR benchmark. |
| Outcome: | The proposed query generator outperforms baselines and existing models on tasks with underexplored intents while using a query generator 47 times smaller than the previous state-of-the-art. |
QueStER: Query Specification for Generative Keyword-Based Retrieval (2026.findings-eacl)
Copied to clipboard
| Challenge: | Generative retrieval (GR) models can be expensive and brittle out of domain. |
| Approach: | They propose a query specification for gEnerative Keyword-Based Retrieval which bridges GR and query reformulation by learning to generate explicit keyword-based search specifications. |
| Outcome: | The proposed query specification improves over existing queries and maintains strong efficiency. |