Towards Robust Neural Retrieval with Source Domain Synthetic Pre-Finetuning (2022.coling-1)
Copied to clipboard
Revanth Gangi Reddy, Vikas Yadav, Md Arafat Sultan, Martin Franz, Vittorio Castelli, Heng Ji, Avirup Sil
| Challenge: | Existing neural IR systems rely on lexical matching for query-passage alignment, while masked language models use a dual encoder architecture to encode passages and questions into continuous vector representations. |
| Approach: | They propose to enhance the out-of-domain generalization of Dense Passage Retrieval (DPR) through synthetic data augmentation only in the source domain. |
| Outcome: | The proposed model outperforms existing models in in-domain and zero-shot evaluations on Wikipedia-based datasets. |
Similar Papers
Dense Passage Retrieval: Is it Retrieving? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) internally store repositories of knowledge, but access to these repositoriels is imprecise. |
| Approach: | They propose a paradigm called retrieval augmented generation to address hallucinations . they analyze the role of fine-tuning pre-trained networks to enhance alignment . |
| Outcome: | The proposed paradigm addresses hallucinations by fine-tuning pre-trained models . the model can be decentralized, inject facts as decentralized representations . |
Relevance-assisted Generation for Robust Zero-shot Retrieval (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Despite strong in-domain performance, dense retrievers have shown poor generalization to out-of-domain zero-shot tasks where no training queries are available. |
| Approach: | They propose to generate domain-specific pseudo queries for fine-tuning with domain-relevant relevance between PQ and documents. |
| Outcome: | The proposed approach is more robust to domain shifts, validated on BEIR zero-shot tasks. |
DPTDR: Deep Prompt Tuning for Dense Passage Retrieval (2022.coling-1)
Copied to clipboard
| Challenge: | Recent studies show that prompt tuning is unfriendly for industrial deployment in dense retrieval tasks. |
| Approach: | They propose to apply prompt tuning to dense retrieval tasks to reduce deployment cost . they propose to use retrieval-oriented intermediate pretraining and unified negative mining . |
| Outcome: | The proposed method outperforms state-of-the-art models on MS-MARCO and Natural Questions. |
Reinforced IR: A Self-Boosting Framework For Domain-Adapted Information Retrieval (2025.acl-long)
Copied to clipboard
| Challenge: | Existing retrieval methods struggle with highly specialized situations that require extensive domain expertise. |
| Approach: | They propose a method that integrates additional information from an LLM-based generator to enhance query performance and train the retriever to better discriminate the relevant documents identified by the generator. |
| Outcome: | The proposed method outperforms existing domain adaptation methods by a large margin and leads to substantial improvements in retrieval quality across a wide range of application scenarios. |
GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval (2022.naacl-main)
Copied to clipboard
| Challenge: | Dense retrieval approaches suffer from the lexical gap and require large amounts of training data. |
| Approach: | They propose an unsupervised method for domain adaptation that uses query generator and pseudo labeling from a cross-encoder to improve retrieval performance. |
| Outcome: | The proposed method outperforms state-of-the-art retrieval methods on domain-specialized datasets by 9.3 points nDCG@10 on six tasks. |
Few-Shot Text Ranking with Meta Adapted Synthetic Weak Supervision (2021.acl-long)
Copied to clipboard
Si Sun, Yingzhuo Qian, Zhenghao Liu, Chenyan Xiong, Kaitao Zhang, Jie Bao, Zhiyuan Liu, Paul Bennett
| Challenge: | Neural information retrieval models have shown advanced results in many ranking scenarios where massive relevance labels or clickthrough data are available. |
| Approach: | They propose a domain adaptive learning method that generalizes Neu-IR models from label-rich source domains to few-shot target domains. |
| Outcome: | The proposed method improves the few-shot ranking accuracy of Neu-IR models on three TREC benchmarks in the web, news, and biomedical domains. |
tRAG: Term-level Retrieval-Augmented Generation for Domain-Adaptive Retrieval (2025.naacl-long)
Copied to clipboard
| Challenge: | Neural retrieval models suffer when there is a domain shift between training and test data distributions. |
| Approach: | They propose to generate domain-adapted pseudo-queries using large language models (LLMs) to improve term recall of unseen query terms by using term-level Retrieval-Augmented Generation (tRAG). |
| Outcome: | The proposed method significantly improves recall for unseen terms by 10.6% and outperforms LLM and retrieval-augmented generation baselines on overall retrieval performance. |
Zero-shot Neural Passage Retrieval via Domain-targeted Synthetic Question Generation (2021.eacl-main)
Copied to clipboard
| Challenge: | Recent advances in neural retrieval have led to advancements on document, passage and knowledge-base benchmarks. |
| Approach: | They propose an approach to zero-shot learning for passage retrieval that uses synthetic question generation to close this gap. |
| Outcome: | The proposed approach can exceed term-based techniques on document retrieval benchmarks by using domain-targeted synthetic question generation. |
Breaking Boundaries in Retrieval Systems: Unsupervised Domain Adaptation with Denoise-Finetuning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing domain adaptation methods for dense retrieval models use unadapted rerank models, leading to imprecise labels. |
| Approach: | They propose to adapt a rerank model to the target domain before using it for label generation. |
| Outcome: | The proposed model achieves better results across three retrieval datasets. |
IR2: Information Regularization for Information Retrieval (2024.lrec-main)
Copied to clipboard
| Challenge: | Effective information retrieval (IR) in settings with limited training data remains a challenging task. |
| Approach: | They propose a technique for reducing overfitting during synthetic data generation . they use DORIS-MAE, ArguAna, and WhatsThatBook as examples . |
| Outcome: | The proposed technique outperforms previous methods and reduces cost by 50% on three recent IR tasks characterized by complex queries. |