| Challenge: | Existing methods for dense retrieval require large supervised datasets with custom hard-negative mining and denoising of positive examples. |
| Approach: | They propose a new corpus-level autoencoding approach for training dense retrieval models that does not require labeled training data. |
| Outcome: | The proposed method matches or surpasses strong supervised performance levels on multiple QA benchmarks with no labeled training data or task-specific losses. |
Similar Papers
Dense Passage Retrieval for Open-Domain Question Answering (2020.emnlp-main)
Copied to clipboard
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, Wen-tau Yih
| Challenge: | Open-domain question answering relies on efficient passage retrieval to select candidate contexts. |
| Approach: | They propose a dual-encoder framework that can be implemented to retrieve passages from a small number of questions and passages. |
| Outcome: | The proposed system outperforms a strong Lucene-BM25 system in top-20 passage retrieval accuracy on multiple open-domain QA benchmarks. |
RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering (2021.naacl-main)
Copied to clipboard
Yingqi Qu, Yuchen Ding, Jing Liu, Kai Liu, Ruiyang Ren, Wayne Xin Zhao, Daxiang Dong, Hua Wu, Haifeng Wang
| Challenge: | Open-domain question answering uses dense passage retrieval to find answers . however, it is difficult to effectively train a dual-encoder due to discrepancy between training and inference . |
| Approach: | They propose an optimized training approach to improve dense passage retrieval using RocketQA . they propose cross-batch negatives, denoised hard negatives and data augmentation . |
| Outcome: | The proposed approach outperforms state-of-the-art models on both MSMARCO and Natural Questions. |
Simple Entity-Centric Questions Challenge Dense Retrievers (2021.emnlp-main)
Copied to clipboard
| Challenge: | Open-domain question answering has exploded in popularity due to the success of dense retrieval models. |
| Approach: | They construct a set of simple, entity-rich questions based on facts from Wikidata and test their models against supervised datasets. |
| Outcome: | The proposed model outperforms sparse retrieval methods on open-domain question answering datasets by a large margin. |
Open Domain Question Answering over Tables via Dense Retrieval (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent advances in open-domain QA focus on retrieving textual passages . a retriever designed to handle tabular context can improve retrieval quality . |
| Approach: | They propose a tabular-based retrieval model that improves retrieval quality over a BERT-based retriever. |
| Outcome: | The proposed retriever improves retrieval quality with mined hard negatives over a BERT-based retriever. |
Evidentiality-aware Retrieval for Overcoming Abstractiveness in Open-Domain Question Answering (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing approaches to ODQA use a simple yet effective retriever-reader framework, but this approach is not always effective in abstractive tasks. |
| Approach: | They propose a method that leverages synthetic distractor samples to learn to discriminate evidence passages from distractors. |
| Outcome: | The proposed method is validated on multiple abstractive open-domain question answering tasks. |
Distantly-Supervised Dense Retrieval Enables Open-Domain Question Answering without Evidence Annotation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Open-domain question answering uses evidence retrieved from large corpus to answer questions . state-of-the-art approaches require intermediate evidence annotations for training . however, such intermediate annotations are expensive and methods that rely on them cannot transfer to the more common setting . |
| Approach: | They propose an open-domain question answering approach that alternately finds evidence from an up-to-date model and encourages the model to learn the most likely evidence. |
| Outcome: | The proposed approach improves over weak retrievers on multi-hop and single-hop benchmarks without using evidence labels. |
Progressively Pretrained Dense Corpus Index for Open-Domain Question Answering (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing open-domain question answering systems are insufficient to capture deep semantic matching that goes beyond lexical overlaps. |
| Approach: | They propose a sample-efficient method to pretrain the paragraph encoder using an existing pretraining model instead of heuristically created pseudo question-paragraph pairs. |
| Outcome: | The proposed method outperforms a strong dense retrieval baseline that uses 6 times more computation for training. |
Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval (2022.acl-long)
Copied to clipboard
| Challenge: | Recent research shows that fine-tuning dense retrievers to realize their capacity requires carefully designed fine-cuning techniques. |
| Approach: | They propose a pre-training architecture that learns to condense information into the dense vector through LM pre-training and a coCondenser architecture which adds an unsupervised corpus-level contrastive loss to warm up the passage embedding space. |
| Outcome: | The proposed architecture reduces the need for heavy data engineering and large batch training. |
Efficient Passage Retrieval with Hashing for Open-domain Question Answering (2021.acl-short)
Copied to clipboard
| Challenge: | Open-domain question answering systems often require large memory to run because of the massive size of their passage index. |
| Approach: | They propose a memory-efficient neural retrieval model that integrates a learning-to-hash technique into the state-of-the-art Dense Passage Retriever to represent the passage index using compact binary codes. |
| Outcome: | The proposed model significantly reduces memory cost from 65GB to 2GB without loss of accuracy on two open-domain question answering benchmarks. |
Dense Hierarchical Retrieval for Open-domain Question Answering (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Dense neural text retrieval has achieved promising results on open-domain Question Answering (QA) current dense retrievers require splitting documents into short passages that usually contain local, partial and sometimes biased context, and may yield inaccurate and misleading hidden representations, thus deteriorating the final retrieval result. |
| Approach: | They propose a hierarchical framework which can generate accurate dense representations of passages by utilizing both macroscopic semantics in the document and microscopic specific to each passage. |
| Outcome: | The proposed framework significantly outperforms the original dense passage retriever and helps an end-to-end QA system outperfect the strong baselines on multiple open-domain QA benchmarks. |