Papers with DPR
Aggretriever: A Simple Approach to Aggregate Textual Representations for Robust Dense Passage Retrieval (2023.tacl-1)
Copied to clipboard
| Challenge: | Pre-trained language models have been successful in knowledge-intensive tasks, but recent research calls into question the robustness of these singlevector models. |
| Approach: | They propose to exploit knowledge in a pre-trained language model for dense passage retrieval by aggregating contextualized token embeddings into a dense vector. |
| Outcome: | The proposed model significantly improves the effectiveness of dense retrieval models on in-domain and zero-shot evaluations without introducing substantial training overhead. |
Multi-Task Dense Retrieval via Model Uncertainty Fusion for Open-Domain Question Answering (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to multitask dense retrieval are not effective due to corpus inconsistency. |
| Approach: | They propose to train individual dense passage retrievers for different open-domain question-answering tasks and aggregate their predictions during test time. |
| Outcome: | The proposed method achieves state-of-the-art performance on 5 benchmark QA datasets, with up to 10% improvement in top-100 accuracy compared to a joint-training multi-task DPR on SQuAD. |
Cross-document Event Coreference Search: Task, Dataset and Modeling (2022.emnlp-main)
Copied to clipboard
| Challenge: | Cross-document Event Coreference resolution is the task of identifying clusters of text mentions that refer to the same event, whether within a single document or across a document collection. |
| Approach: | They propose a cross-document coreference search task that searches for all coreferring mentions for a query event in a large document collection. |
| Outcome: | The proposed model integrates a powerful coreference scoring scheme into the DPR architecture, yielding improved performance. |
Exploring Contrast Consistency of Open-Domain Question Answering Systems on Minimally Edited Questions (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing studies on the ability of a model to make consistently correct predictions in the presence of perturbations have not been conducted in open-domain question answering (OpenQA). |
| Approach: | They propose a query-side contrastive loss to improve the dense passage retriever (DPR) to improve DPR training. |
| Outcome: | The proposed approach improves the density of the dense passage retriever (DPR) training set without sacrificing accuracy on standard test sets. |
Towards Robust Neural Retrieval with Source Domain Synthetic Pre-Finetuning (2022.coling-1)
Copied to clipboard
Revanth Gangi Reddy, Vikas Yadav, Md Arafat Sultan, Martin Franz, Vittorio Castelli, Heng Ji, Avirup Sil
| Challenge: | Existing neural IR systems rely on lexical matching for query-passage alignment, while masked language models use a dual encoder architecture to encode passages and questions into continuous vector representations. |
| Approach: | They propose to enhance the out-of-domain generalization of Dense Passage Retrieval (DPR) through synthetic data augmentation only in the source domain. |
| Outcome: | The proposed model outperforms existing models in in-domain and zero-shot evaluations on Wikipedia-based datasets. |
Dense Template Retrieval for Customer Support (2022.coling-1)
Copied to clipboard
| Challenge: | Templated answers are used to cover a wide range of topics, but the number of templates is often too high for an agent to manually search. |
| Approach: | They propose a dense retrieval framework that adapts a standard in-batch negatives technique to support unpaired sampling of queries and templates. |
| Outcome: | The proposed approach improves performance and training speed over more standard methods. |
Self-Bootstrapped Visual-Language Model for Knowledge Selection and Question Answering (2024.emnlp-main)
Copied to clipboard
| Challenge: | a framework that leverages the visual-language model to select key knowledge retrieved by DPR and answer questions improves performance of the baseline on the open-domain Knowledge-based VQA benchmark, OK-VQA. |
| Approach: | They propose a framework that leverages visual-language models to retrieve related knowledge . they use dense passage retrieval to retrieve knowledge related to visual-linguistics . |
| Outcome: | The proposed framework significantly improves the baseline on the open-domain Knowledge-based VQA benchmark, OK-VQA. |
Efficient Passage Retrieval with Hashing for Open-domain Question Answering (2021.acl-short)
Copied to clipboard
| Challenge: | Open-domain question answering systems often require large memory to run because of the massive size of their passage index. |
| Approach: | They propose a memory-efficient neural retrieval model that integrates a learning-to-hash technique into the state-of-the-art Dense Passage Retriever to represent the passage index using compact binary codes. |
| Outcome: | The proposed model significantly reduces memory cost from 65GB to 2GB without loss of accuracy on two open-domain question answering benchmarks. |
A Joint Model for Dropped Pronoun Recovery and Conversational Discourse Parsing in Chinese Conversational Speech (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work regards dropped pronoun recovery and conversational discourse parsing as two separate tasks and tackles them separately. |
| Approach: | They propose a neural model for dropped pronoun recovery and conversational discourse parsing in Chinese conversational speech. |
| Outcome: | The proposed model outperforms the state-of-the-art models on a new dataset . the proposed model is based on linguistic and semantic information from Chinese conversational speech . |
A Study on the Efficiency and Generalization of Light Hybrid Retrievers (2023.acl-short)
Copied to clipboard
Man Luo, Shashank Jain, Anchit Gupta, Arash Einolghozati, Barlas Oguz, Debojeet Chatterjee, Xilun Chen, Chitta Baral, Peyman Heidari
| Challenge: | Recent research focuses on building neural retrievers which learn dense embeddings of query and document into a semantic space. |
| Approach: | They propose to use an indexing-efficient dense retriever to reduce hybrid retrievers' memory by using the state-based indexing algorithm. |
| Outcome: | The proposed hybrid retriever saves 13 memory while maintaining 98.0% performance on out-of-domain datasets and adversarial attacks datasets. |
CRAFT: Training-Free Cascaded Retrieval for Tabular QA (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for open-domain table question answering require retraining or fine-tuning on new datasets. |
| Approach: | They propose a zero-shot, cascaded retrieval approach that uses a sparse retrieval model to filter a subset of candidates before applying more expensive dense models as re-rankers. |
| Outcome: | The proposed method outperforms state-of-the-art retrieval models on the NQ-Tables dataset. |
Learning to Retrieve Passages without Supervision (2022.naacl-main)
Copied to clipboard
| Challenge: | Dense retrievers for open domain question answering have been shown to achieve impressive performance by training on large datasets of question-passage pairs. |
| Approach: | They propose to use recurring spans to create pseudo examples for contrastive learning. |
| Outcome: | The proposed model outperforms all pretrained baselines on a wide range of ODQA datasets and is competitive with BM25, a strong sparse baseline. |
Open-Domain Safety Policy Construction (2026.findings-eacl)
Copied to clipboard
| Challenge: | Moderation layers are core component of many products built on user-generated content. |
| Approach: | They propose a system that drafts a content moderation policy based on human-written seed domain information. |
| Outcome: | The proposed system outperforms definition-only and in-context learning baselines on openAI undesired content benchmarks and an in-house multimodal advertisement moderation benchmark. |
Relation-Guided Pre-Training for Open-Domain Question Answering (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing QA datasets are imbalanced in some types of relations, which hurts generalization performance over long-tail questions. |
| Approach: | They propose a relation-guided pre-training framework to infer latent relations from a QA dataset . they then propose RGPT-QA to conduct extractive QA to get the target answer entity . |
| Outcome: | The proposed framework improves Exact Match accuracy on natural questions, TriviaQA, and WebQuestions. |
Adaptive Information Seeking for Open-Domain Question Answering (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing iterative approaches to open-domain question answering use predefined strategies . e.g., BM25, DPR, and hyperlink are defined as actions . |
| Approach: | They propose a novel adaptive information-seeking strategy for open-domain question answering . they propose to use a partially observed Markov decision process to select a proper retrieval action . |
| Outcome: | Experiments on SQuAD Open and HotpotQA fullwiki show that AISO outperforms baseline methods with predefined strategies in retrieval and answer evaluations. |
Generation-Augmented Retrieval for Open-Domain Question Answering (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to answer open-domain questions use sparse representations and sparsity. |
| Approach: | They propose a method which augments a query by generating relevant contexts from heuristically discovered contexts without external supervision. |
| Outcome: | The proposed approach outperforms state-of-the-art dense retrieval methods on natural questions and triviaQA datasets. |
MTA4DPR: Multi-Teaching-Assistants Based Iterative Knowledge Distillation for Dense Passage Retrieval (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown the effectiveness of knowledge distillation in DPR, but there is a performance gap between the teacher and the distilled student. |
| Approach: | They propose an iterative knowledge distillation method which transfers knowledge from teacher to student with help of multiple assistants in an iterated manner. |
| Outcome: | The proposed method achieves state-of-the-art performance among models with same parameters on multiple datasets and is competitive when compared with larger models. |
Meta-training with Demonstration Retrieval for Efficient Few-shot Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Large language models have impressive fewshot performance on many NLP tasks and domains. |
| Approach: | They propose a meta-training approach that uses demonstration retrieval to train parameter-efficient models that generalize well on a larger variety of tasks. |
| Outcome: | The proposed approach outperforms many parameter-efficient methods on QA, NLI, and text classification tasks. |
Strong and Efficient Baselines for Open Domain Conversational Question Answering (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Current Open Domain Question Answering (ODQA) approaches to reevaluating baselines for efficiency and effectiveness have been limited due to various limitations. |
| Approach: | They propose and evaluate strong yet simple and efficient baselines by introducing a fast reranking component between the retriever and the reader and performing targeted finetuning steps. |
| Outcome: | The proposed method improves the SotA results while reducing the reader’s latency by 60%. |
WikiCREM: A Large Unsupervised Corpus for Coreference Resolution (D19-1)
Copied to clipboard
| Challenge: | Large-scale training sets for pronoun resolution are scarce, since manually labelling data is costly. |
| Approach: | They propose a language-model-based approach to solve pronoun disambiguation problems using a WikiCREM dataset. |
| Outcome: | The proposed model outperforms state-of-the-art approaches on 6 out of 7 datasets. |
Hyperlink-induced Pre-training for Passage Retrieval in Open-domain Question Answering (2022.acl-long)
Copied to clipboard
Jiawei Zhou, Xiaoguang Li, Lifeng Shang, Lan Luo, Ke Zhan, Enrui Hu, Xinyu Zhang, Hao Jiang, Zhao Cao, Fan Yu, Xin Jiang, Qun Liu, Lei Chen
| Challenge: | Existing methods to train dense passage retrieval have a large data gap between upstream and downstream relevance. |
| Approach: | They propose a method to pre-train the dense retriever with the text relevance induced by hyperlinks within Web documents. |
| Outcome: | The proposed method outperforms existing methods under different scenarios and in the open-domain question answering domain. |
End-to-End Training of Neural Retrievers for Open-Domain Question Answering (2021.acl-long)
Copied to clipboard
Devendra Sachan, Mostofa Patwary, Mohammad Shoeybi, Neel Kant, Wei Ping, William L. Hamilton, Bryan Catanzaro
| Challenge: | Recent work on training neural retrievers for open-domain question answering (OpenQA) has employed both supervised and unsupervised methods. |
| Approach: | They propose an approach of unsupervised pre-training with the Inverse Cloze Task and masked salient spans followed by supervised finetuning using question-context pairs. |
| Outcome: | The proposed approach outperforms models like REALM and RAG in retrieval accuracy and answer extraction. |
CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrieval (2023.acl-long)
Copied to clipboard
Minghan Li, Sheng-Chieh Lin, Barlas Oguz, Asish Ghoshal, Jimmy Lin, Yashar Mehdad, Wen-tau Yih, Xilun Chen
| Challenge: | Existing multi-vector retrieval methods are slower and require more space to store indices compared to their single-vektor counterparts. |
| Approach: | They propose a multi-vector retrieval method that uses dynamic lexical routing to route different token vectors to the predicted lexicals. |
| Outcome: | The proposed method achieves state-of-the-art performance on several benchmark datasets while being nearly 40 times faster than the current state-out-of the-art method. |
Hybrid Hierarchical Retrieval for Open-Domain Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work shows that dense hierarchical retrieval (DHR) can outperform dense passage retrieval. |
| Approach: | They propose a framework that applies sparse, dense and a combination of them to document and passage retrieval. |
| Outcome: | The proposed framework can outperform dense hierarchical retrieval (DHR) and sparse retrievers (BM25) on open-domain question answering (ODQA) datasets with an average improvement of 4.69% on recall@100 over DHR. |
Expand, Rerank, and Retrieve: Query Reranking for Open-Domain Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | Empirically, EAR improves top-5/20 accuracy by 3-8 and 5-10 points . dense retrievers are limited by their inability to perform semantic matching for relevant passages that have low lexical overlap with the query. |
| Approach: | They propose a query expansion and reranking approach for improving passage retrieval with the application to open-domain question answering. |
| Outcome: | Empirically, EAR improves top-5/20 accuracy by 3-8 and 5-10 points when compared to a vanilla query expansion model and a dense retrieval model. |
Retrieval Augmented Visual Question Answering with Outside Knowledge (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent systems that use Dense Passage Retrieval (DPR) for visual question answering use Denses trained separately from answer generation. |
| Approach: | They propose a joint training scheme that integrates DPR with answer generation to improve the performance of OK-VQA systems. |
| Outcome: | The proposed model outperforms existing systems with strong Dense Passage Retrieval (DPR) it yields significant improvements in answer quality and computation required for training. |
Dense Passage Retrieval: Is it Retrieving? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) internally store repositories of knowledge, but access to these repositoriels is imprecise. |
| Approach: | They propose a paradigm called retrieval augmented generation to address hallucinations . they analyze the role of fine-tuning pre-trained networks to enhance alignment . |
| Outcome: | The proposed paradigm addresses hallucinations by fine-tuning pre-trained models . the model can be decentralized, inject facts as decentralized representations . |
Can Retriever-Augmented Language Models Reason? The Blame Game Between the Retriever and the Language Model (2023.findings-emnlp)
Copied to clipboard
| Challenge: | kNN-LM, REALM, DPR + FiD, Contriever + ATLAS, and Contriver + Flan-T5 are popular retriever-augmented language models for a variety of tasks. |
| Approach: | They evaluate the strengths and weaknesses of kNN-LM, REALM, DPR + FiD, Contriever + ATLAS, and Contriver + Flan-T5 in reasoning over retrieved statements across different tasks. |
| Outcome: | The proposed models do not exhibit strong reasoning even when provided with only the required statements. |
Decoding Dense Embeddings: Sparse Autoencoders for Interpreting and Discretizing Dense Retrieval (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing sparse retrieval methods suffer from a lack of interpretability . we propose a new interpretability framework that decomposes dense embeddings into distinct, interpretable latent concepts. |
| Approach: | They propose a new interpretability framework that leveragesSparse Autoencoders to decompose uninterpretable dense embeddings fromDPR models into distinct, interpretable latent concepts. |
| Outcome: | The proposed interpretability framework achieves high index-space and computational efficiency while maintaining robust performance across vocabulary and semantic mismatches. |
MA-DPR: Manifold-aware Distance Metrics for Dense Passage Retrieval (2025.emnlp-main)
Copied to clipboard
| Challenge: | Empirical evidence suggests that manifold-aware distance allows DPR to leverage context from related neighboring passages. |
| Approach: | They propose a "manifold-aware" distance metric that measures query–passage distance . they propose to model the intrinsic manifold structure of passages using a nearest-neighbor graph . |
| Outcome: | Empirical evidence suggests MA-DPR outperforms Euclidean and cosine distances on OOD . it can be applied to a wide range of dense embedding and retrieval tasks . |