Efficient Passage Retrieval with Hashing for Open-domain Question Answering (2021.acl-short)
Copied to clipboard
| Challenge: | Open-domain question answering systems often require large memory to run because of the massive size of their passage index. |
| Approach: | They propose a memory-efficient neural retrieval model that integrates a learning-to-hash technique into the state-of-the-art Dense Passage Retriever to represent the passage index using compact binary codes. |
| Outcome: | The proposed model significantly reduces memory cost from 65GB to 2GB without loss of accuracy on two open-domain question answering benchmarks. |
Similar Papers
Dense Passage Retrieval for Open-Domain Question Answering (2020.emnlp-main)
Copied to clipboard
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, Wen-tau Yih
| Challenge: | Open-domain question answering relies on efficient passage retrieval to select candidate contexts. |
| Approach: | They propose a dual-encoder framework that can be implemented to retrieve passages from a small number of questions and passages. |
| Outcome: | The proposed system outperforms a strong Lucene-BM25 system in top-20 passage retrieval accuracy on multiple open-domain QA benchmarks. |
Open Domain Question Answering over Tables via Dense Retrieval (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent advances in open-domain QA focus on retrieving textual passages . a retriever designed to handle tabular context can improve retrieval quality . |
| Approach: | They propose a tabular-based retrieval model that improves retrieval quality over a BERT-based retriever. |
| Outcome: | The proposed retriever improves retrieval quality with mined hard negatives over a BERT-based retriever. |
Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing approaches to extracting answer from text are expensive to train and train. |
| Approach: | They investigate how much models benefit from retrieving text passages . they obtain state-of-the-art results on the Natural Questions and TriviaQA open benchmarks ." |
| Outcome: | The proposed model performs better when retrieving more passages than previously thought . |
Silver Retriever: Advancing Neural Passage Retrieval for Polish Question Answering (2024.lrec-main)
Copied to clipboard
| Challenge: | lexical approaches to find passages have outperformed lexicals due to their superior performance . however, for some languages, such as Polish, few models are available . a recent study shows that neural retrievers are more efficient and efficient than lexica. |
| Approach: | They present a neural retriever for Polish trained on a diverse collection of manual and weakly labeled datasets. |
| Outcome: | The proposed model outperforms lexical retrieval models in Polish on three retrieval tasks. |
FastFiD: Improve Inference Efficiency of Open Domain Question Answering via Sentence Selection (2024.acl-long)
Copied to clipboard
| Challenge: | Open Domain Question Answering (ODQA) is a longstanding task in Natural Language Processing that involves generating an answer solely based on a given question. |
| Approach: | They propose a novel approach that executes sentence selection on the encoded passages to enhance the inference speed while reducing the context length required for generating answers. |
| Outcome: | The proposed approach can increase inference speed by **2.3X-5.7X** while maintaining the model’s performance. |
End-to-End Training of Neural Retrievers for Open-Domain Question Answering (2021.acl-long)
Copied to clipboard
Devendra Sachan, Mostofa Patwary, Mohammad Shoeybi, Neel Kant, Wei Ping, William L. Hamilton, Bryan Catanzaro
| Challenge: | Recent work on training neural retrievers for open-domain question answering (OpenQA) has employed both supervised and unsupervised methods. |
| Approach: | They propose an approach of unsupervised pre-training with the Inverse Cloze Task and masked salient spans followed by supervised finetuning using question-context pairs. |
| Outcome: | The proposed approach outperforms models like REALM and RAG in retrieval accuracy and answer extraction. |
Dense Hierarchical Retrieval for Open-domain Question Answering (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Dense neural text retrieval has achieved promising results on open-domain Question Answering (QA) current dense retrievers require splitting documents into short passages that usually contain local, partial and sometimes biased context, and may yield inaccurate and misleading hidden representations, thus deteriorating the final retrieval result. |
| Approach: | They propose a hierarchical framework which can generate accurate dense representations of passages by utilizing both macroscopic semantics in the document and microscopic specific to each passage. |
| Outcome: | The proposed framework significantly outperforms the original dense passage retriever and helps an end-to-end QA system outperfect the strong baselines on multiple open-domain QA benchmarks. |
Open-Domain Question Answering with Pre-Constructed Question Spaces (2021.naacl-srw)
Copied to clipboard
| Challenge: | Open-domain question answering aims at locating answers to user-generated questions in massive collections of documents. |
| Approach: | They propose an algorithm with a novel reader-retriever design that differs from both families of algorithms. |
| Outcome: | The proposed algorithm outperforms retrieval-based methods with two large-scale datasets and is state-of-the-art. |
You Only Need One Model for Open-domain Question Answering (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent approaches to Open-domain Question Answering use external knowledge bases, but have separate parameters and are weakly-coupled during training. |
| Approach: | They propose to use a single question answering model trained end-to-end to retrieve external knowledge and rerank passages with a separate reranked model. |
| Outcome: | The proposed model outperforms the previous state-of-the-art model by 1.0 and 0.7 exact match scores on the Natural Questions and TriviaQA open datasets. |
RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering (2021.naacl-main)
Copied to clipboard
Yingqi Qu, Yuchen Ding, Jing Liu, Kai Liu, Ruiyang Ren, Wayne Xin Zhao, Daxiang Dong, Hua Wu, Haifeng Wang
| Challenge: | Open-domain question answering uses dense passage retrieval to find answers . however, it is difficult to effectively train a dual-encoder due to discrepancy between training and inference . |
| Approach: | They propose an optimized training approach to improve dense passage retrieval using RocketQA . they propose cross-batch negatives, denoised hard negatives and data augmentation . |
| Outcome: | The proposed approach outperforms state-of-the-art models on both MSMARCO and Natural Questions. |