Episodic Memory Reader: Learning What to Remember for Question Answering from Streaming Data (P19-1)
Copied to clipboard
| Challenge: | Existing QA methods lack scalability and performance is difficult to solve with document-level contexts. |
| Approach: | They propose an end-to-end deep network model that sequentially reads the input contexts into an external memory while replacing memories that are less important for answering unseen questions. |
| Outcome: | The proposed model improves on a synthetic dataset and real-world large-scale textual and video QA datasets. |
Similar Papers
Beyond Markovian Forgetfulness: Episodic Memory for Reasoning-Intensive Retrieval (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for reasoning-intensive information retrieval suffer from inefficiency . Chain-of-Thought (CoT) approaches suffer from lack of token efficiency . Existing models lack episodic memory, which stores the history of prior states . |
| Approach: | They propose an algorithm that enhances state-based frameworks with an episodic memory module that stores the full history of prior states for a query. |
| Outcome: | The proposed model outperforms CoT and state-based models on the BRIGHT benchmark and is highly token-efficient. |
A Memory Model for Question Answering from Streaming Data Supported by Rehearsal and Anticipation of Coreference Information (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing question answering methods assume that the input content can always be accessed while answering the question. |
| Approach: | They propose a model that performs rehearsal and anticipation while processing inputs to memorize important information for question answering tasks from streaming data. |
| Outcome: | The proposed model improves on short-sequence (bAbI) and large-squence textual (NarrativeQA) and video (ActivityNet-QA) question answering datasets. |
Retrieval as Attention: End-to-end Learning of Retrieval and Reading within a Single Transformer (2022.emnlp-main)
Copied to clipboard
| Challenge: | eschewing separate architecture and training for knowledge-intensive tasks is cumbersome . end-to-end training only based on supervision from the end task is awkward . |
| Approach: | They propose a single Transformer that performs retrieval as attention and end-to-end training solely based on supervision from the end QA task. |
| Outcome: | The proposed model outperforms state-of-the-art retrievers and readers on in-domain datasets. |
MemoReader: Large-Scale Reading Comprehension through Neural Memory Controller (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to machine reading comprehension are limited in understanding, up to a few paragraphs, failing to comprehend lengthy documents. |
| Approach: | They propose a deep neural network architecture to handle a long-range dependency in RC tasks. |
| Outcome: | The proposed method outperforms existing methods especially for lengthy documents. |
An Efficient Memory-Augmented Transformer for Knowledge-Intensive NLP Tasks (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods rely on parametric models that store knowledge in parameters or retrieval-augmented models that have access to external knowledge sources. |
| Approach: | They propose a parametric parametric model that stores knowledge in its parameters or a retrieval-augmented model that has access to external knowledge sources. |
| Outcome: | The proposed method runs substantially faster across the board and produces more accurate results on WoW and ELI5. |
EpMAN: Episodic Memory AttentioN for Generalizing to Longer Contexts (2025.acl-long)
Copied to clipboard
Subhajit Chaudhury, Payel Das, Sarathkrishna Swaminathan, Georgios Kollias, Elliot Nelson, Khushbu Pahwa, Tejaswini Pedapati, Igor Melnyk, Matthew Riemer
| Challenge: | Recent advances in Large Language Models (LLMs) have yielded impressive successes on many language tasks, but efficient processing of long contexts remains a significant challenge. |
| Approach: | They propose a method for processing long contexts in an episodic memory module while holistically attending to semantically-relevant context chunks. |
| Outcome: | The proposed method outperforms baseline decoders on multiple long-context recall and question-answering benchmarks on 16k to 256k tokens. |
Cut to the Chase: A Context Zoom-in Network for Reading Comprehension (D18-1)
Copied to clipboard
| Challenge: | Recent deep-learning based models suffer from reasoning over long documents and do not trivially generalize to cases where the answer is not present as a span. |
| Approach: | They propose a novel context zoom-in network (ConZNet) that can skip through irrelevant parts of a document and generate an answer using only the relevant regions of text. |
| Outcome: | The proposed architecture outperforms state-of-the-art results by 12.62% (ROUGE-L) relative improvement on the recently proposed and challenging RC dataset ‘NarrativeQA’. |
Enhancing Key-Value Memory Neural Networks for Knowledge Based Question Answering (N19-1)
Copied to clipboard
| Challenge: | Existing Key-value Memory Neural Networks are effective for shallow reasoning over documents . but extending them to Knowledge Based Question Answering is not trivial . |
| Approach: | They propose a mechanism to enable conventional KV-MemNNs models to perform interpretable reasoning for complex questions. |
| Outcome: | The proposed solution provides better reasoning abilities on complex questions and achieves state-of-the-art performance. |
ReadTwice: Reading Very Large Documents with Memories (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to model long-range dependencies in text are limited to 512 tokens . however, the amount of compute in attention depends quadratically on the number of tokens in an input text passage. |
| Approach: | They propose a technique that summarises text into a memory table to be used in a second read of the text. |
| Outcome: | The proposed method outperforms models of comparable size on several question answering datasets and sets a new state of the art on the NarrativeQA task, with questions about entire books. |
Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for open-domain question-answering use an open book approach . a recent alternative is to retrieve from a collection of previously-generated question-annwer pairs . |
| Approach: | They propose a new QA system that augments a text-to-text model with a large memory of question-answer pairs and a task for the latent step of question retrieval. |
| Outcome: | The proposed system outperforms closed-book QA and can answer multi-hop questions. |