HIL: Hybrid Isotropy Learning for Zero-shot Performance in Dense retrieval (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent advances in dense retrieval models have brought ColBERT to prominence in information retrieval, but it is underperforming in zero-shot tasks. |
| Approach: | They propose a Hybrid Isotropy Learning architecture that integrates isotropic and anisotropic representations to improve zero-shot retrieval performance. |
| Outcome: | The proposed model outperforms the baseline ColBERT model in BEIR benchmarks. |
Similar Papers
ColBERT-XM: A Modular Multi-Vector Representation Model for Zero-Shot Multilingual Information Retrieval (2025.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to improve retrieval effectiveness in high-resource languages are limited due to the lack of high-quality labeled data in non-English languages. |
| Approach: | They propose a modular dense retrieval model that learns from the rich data of a single high-resource language and effectively zero-shot transfers to a wide array of languages. |
| Outcome: | The proposed model performs well against state-of-the-art multilingual retrieval models trained on more extensive datasets in various languages. |
Query-focused Referentiability Learning for Zero-shot Retrieval (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing dense representations have shown limitations in zero-shot scenarios . however, passage representations fail to align with their gold queries . |
| Approach: | They propose a query-focused concept of 'referentiable' which ensures passage representations are referenced by their gold queries. |
| Outcome: | The proposed model outperforms existing models on the BEIR benchmark. |
Isotropic Representation Can Improve Zero-Shot Cross-Lingual Transfer on Multilingual Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing mPLMs can align representations well for myriads of cross-lingual transfer tasks. |
| Approach: | They propose enhanced isotropy and constrained code-switching for zero-shot cross-lingual transfer to alleviate the problem of misalignment caused by anisotropic representations. |
| Outcome: | The proposed method improves on three zero-shot cross-lingual transfer tasks and over existing methods. |
A Thorough Examination on Zero-shot Dense Retrieval (2023.findings-emnlp)
Copied to clipboard
Ruiyang Ren, Yingqi Qu, Jing Liu, Xin Zhao, Qifei Wu, Yuchen Ding, Hua Wu, Haifeng Wang, Ji-Rong Wen
| Challenge: | Recent advances in dense retrieval (DR) models have been shown to be not as competitive as traditional sparse retrieval models in a zero-shot retrieval setting. |
| Approach: | They propose to examine the zero-shot capability of DR models by analyzing key factors related to source training set and potential bias from target dataset. |
| Outcome: | The proposed model is not as competitive as sparse retrieval models in a zero-shot retrieval setting. |
Zero-Shot Dense Retrieval with Momentum Adversarial Domain Invariant Representations (2022.findings-acl)
Copied to clipboard
| Challenge: | Dense retrieval (DR) methods first encode texts into a dense embedding space and then conduct text retrieval using efficient nearest neighbor search. |
| Approach: | They propose Momentum adversarial Domain Invariant Representation learning to train a domain classifier that distinguishes source versus target domains and adversarially updates the DR encoder to learn domain invariant representations. |
| Outcome: | The proposed method outperforms baselines on 10+ ranking datasets collected in the BEIR benchmark in the zero-shot setting, with more than 10% relative gains on datasets with enough sensitivity for DR models’ evaluation. |
A Representation Sharpening Framework for Zero Shot Dense Retrieval (2026.eacl-long)
Copied to clipboard
| Challenge: | Zero-shot dense retrieval requires generic, pretrained DRs, which struggle to represent semantic differences between similar documents. |
| Approach: | They propose a training-free representation sharpening framework that augments a document’s representation with information that helps differentiate it from similar documents in the corpus. |
| Outcome: | The proposed framework is compatible with prior approaches to zero-shot dense retrieval and consistently improves their performance. |
Few-shot Prompting for Pairwise Ranking: An Effective Non-Parametric Retrieval Model (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies show that training examples improve zero-shot performance of supervised ranking models. |
| Approach: | They propose to augment supervised ranking models with pairs of queries and documents to improve their performance. |
| Outcome: | The proposed model outperforms the unsupervised models on in-domain and out-domain retrieval benchmarks. |
Precise Zero-Shot Dense Retrieval without Relevance Labels (2023.acl-long)
Copied to clipboard
| Challenge: | Existing dense retrieval systems that use semantic embedding similarities can be effective across tasks and languages. |
| Approach: | They propose to pivot through Hypothetical Document Embeddings (HyDE) given a query, HyDE first zero-shot prompts an instruction-following language model to generate a hypothetical document. |
| Outcome: | The proposed method significantly outperforms the state-of-the-art unsupervised dense retriever Contriever and shows strong performance comparable to fine-tuned retrievers across tasks and languages. |
ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction (2022.naacl-main)
Copied to clipboard
| Challenge: | Neural information retrieval (IR) methods encode queries and documents into single vectors, but late interaction models produce multi-vector representations at the granularity of each token. |
| Approach: | They propose a retrieval method that couples an aggressive residual compression mechanism with a denoised supervision strategy to improve the quality and space footprint of late interaction. |
| Outcome: | The proposed retriever improves quality and space footprint of late interaction models while reducing space footprint by 6–10x. |
Retrieval-Augmented Retrieval: Large Language Models are Strong Zero-Shot Retriever (2024.findings-acl)
Copied to clipboard
Tao Shen, Guodong Long, Xiubo Geng, Chongyang Tao, Yibin Lei, Tianyi Zhou, Michael Blumenstein, Daxin Jiang
| Challenge: | Large-scale retrieval is indispensable in information-seeking tasks such as open-domain question answering and knowledgegrounded dialogue. |
| Approach: | They propose to use a large language model (LLM) to augment a query with its potential answers by prompting LLMs with a composition of the query and the query’s in-domain candidates. |
| Outcome: | The proposed method breaks brute-force combinations of retrievers with LLMs and lifts the performance of zero-shot retrieval to be very competitive on benchmark datasets. |