Hyunji Lee, JaeYoung Kim, Hoyeon Chang, Hanseok Oh, Sohee Yang, Vladimir Karpukhin, Yi Lu, Minjoon Seo
| Challenge: | Existing text retrieval models depend on the information encoded in its parameters without external memory, its information capacity is limited and fixed. |
| Approach: | They propose a nonparametric decoding approach which uses external memory instead of vanilla vocab embeddings as decoder voka embedds. |
| Outcome: | The proposed model can utilize parametric and nonparametric space. |
Similar Papers
DiffusionRet: Diffusion-Enhanced Generative Retriever using Constrained Decoding (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Generative retrieval methods have suffered from the lack of the intermediate reasoning step . generative retrieval uses sequence-to-sequence diffusion models to map a query to relevant docids . |
| Approach: | They propose a novel method that uses query as an intermediate step before retrieval . they propose to use sequence-to-sequence diffusion models to map a query to relevant docids . |
| Outcome: | Experiments show that proposed method outperforms existing methods on MARCO and Natural Questions datasets. |
Re3val: Reinforced and Reranked Generative Retrieval (2024.findings-eacl)
Copied to clipboard
| Challenge: | generative retrieval models encode pointers to information in a corpus as an index within the model’s parameters. |
| Approach: | They propose a generative retrieval model that leverages contextual information to rerank retrieved page titles and utilizes REINFORCE to maximize rewards generated by constrained decoding. |
| Outcome: | The proposed model can't be tuned for the downstream readers as decoding the page title is a non-differentiable operation. |
Generative Dense Retrieval: Memory Can Be a Burden (2024.eacl-long)
Copied to clipboard
| Challenge: | Empirical results show that Generative Dense Retrieval (GDR) achieves an average of 3.0 R@100 improvement on NQ dataset under multiple settings and has better scalability. |
| Approach: | They propose a Generative Dense Retrieval paradigm that auto-decodes document identifiers given a query and uses memory to avoid memory confusion. |
| Outcome: | Empirical results show that the proposed paradigm improves on the small-scale corpora and improves scalability. |
GLEN: Generative Retrieval via Lexical Index Learning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for document retrieval bypass auxiliary index structures and can be optimized through end-to-end learning. |
| Approach: | They propose a method to generate a relevant document's identifier using an index learning strategy. |
| Outcome: | The proposed method achieves state-of-the-art or competitive performance on benchmark datasets. |
Bag-of-Words Transfer: Non-Contextual Techniques for Multi-Task Learning (D19-61)
Copied to clipboard
| Challenge: | Existing approaches to multi-task learning take advantage of transfer among tasks . generative reconstruction of the observations is not included in the standard framework . |
| Approach: | They propose to use a syntactically-oblivious pooling encoder and pre-trained word embeddings to improve sentence-level representations. |
| Outcome: | The proposed techniques yield similar performance on a universe of task combinations while reducing training time and model size. |
NAG-NER: a Unified Non-Autoregressive Generation Framework for Various NER Tasks (2023.acl-industry)
Copied to clipboard
| Challenge: | Existing models for general NER tasks require entities to be generated in a predefined order, causing error propagation and inefficient decoding. |
| Approach: | They propose a non-autoregressive generation framework for general NER tasks that generates entities as a set instead of a sequence, avoiding error propagation and inefficient decoding. |
| Outcome: | The proposed model outperforms state-of-the-art models on three benchmark NER datasets and two of our proprietary NER tasks. |
Augmenting Document Representations for Dense Retrieval with Interpolation and Perturbation (2022.acl-short)
Copied to clipboard
| Challenge: | Existing sparse retrieval models rely on term-based matching to retrieve relevant documents. |
| Approach: | They propose a framework which augments the representations of documents with interpolation and perturbation. |
| Outcome: | The proposed framework significantly outperforms baselines on the dense retrieval of both the labeled and unlabeled documents. |
D3: Dynamic Docid Decoding for Multi-Intent Generative Retrieval (2026.eacl-industry)
Copied to clipboard
| Challenge: | Existing GR systems rely on offline DocID assignment and constrained decoding . offline Doc ID assignment and decoding often prevents GR from capturing query-specific intent . |
| Approach: | They propose a mechanism that adaptively refines DocIDs through query-informed identifier expansion. |
| Outcome: | The proposed mechanism improves retrieval accuracy on unseen and multi-intent documents. |
Entropy-Based Decoding for Retrieval-Augmented Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Despite their success, retrieval-augmented LLMs still face the distractibility issue, where the generated responses are negatively influenced by noise from both external and intrinsic knowledge sources. |
| Approach: | They propose a entropy-based document-parallel ensemble decoding method that prioritizes low-entropies from retrieved documents and incorporates a contrastive decoding mechanism that contrasts the obtained low- and high-entropic ensemble distributions with the high-end internal knowledge across layers. |
| Outcome: | The proposed method improves on open-domain question answering datasets and shows that it is highly efficient. |
Pseudo-Relevance for Enhancing Document Representation (2022.emnlp-main)
Copied to clipboard
| Challenge: | a novel approach to document retrieval can be used to encode documents as vectors . a few query-relevant terms can be pruned out to reduce index overhead . |
| Approach: | They propose to enhance the document representation for the bi-encoder approach in dense document retrieval. |
| Outcome: | The proposed solution reduces latency and memory footprint up to 8- and 3-fold . it is validated on MSMARCO and real-world search query logs . |