Working Memory Networks: Augmenting Memory Networks with a Relational Reasoning Module (P18-1)
Copied to clipboard
| Challenge: | Recent advances in deep neural networks have enabled complex reasoning tasks. |
| Approach: | They propose a MemNN architecture with a working memory storage and reasoning module that retains relational reasoning abilities of relation networks while reducing computational complexity. |
| Outcome: | The proposed model retains the relational reasoning abilities of the RN while reducing its computational complexity from quadratic to linear. |
Similar Papers
Enhancing Key-Value Memory Neural Networks for Knowledge Based Question Answering (N19-1)
Copied to clipboard
| Challenge: | Existing Key-value Memory Neural Networks are effective for shallow reasoning over documents . but extending them to Knowledge Based Question Answering is not trivial . |
| Approach: | They propose a mechanism to enable conventional KV-MemNNs models to perform interpretable reasoning for complex questions. |
| Outcome: | The proposed solution provides better reasoning abilities on complex questions and achieves state-of-the-art performance. |
Text Modular Networks: Learning to Decompose Tasks in the Language of Existing Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to decompose complex tasks into simpler ones do not require annotated decompositions. |
| Approach: | They propose a framework for building interpretable systems that learn to solve complex tasks by decomposing existing models into simpler ones solvable by existing models. |
| Outcome: | The proposed framework is more versatile than existing explainable systems for DROP and HotpotQA datasets, is more robust than state-of-the-art blackbox (uninterpretable) systems, and generates more understandable and trustworthy explanations compared to prior work. |
Relational Memory-Augmented Language Models (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing language models rely on word correlation and are difficult to interpret . existing models often lack explicit representations for such information . |
| Approach: | They propose a memory-augmented approach to condition autoregressive language models on knowledge graphs. |
| Outcome: | The proposed model improves perplexity and bits per character in an autoregressive language model . it is complementary to token-based memory and enables causal interventions . |
Towards Enhancing Relational Rules for Knowledge Graph Link Prediction (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge graph reasoning methods are inadequate for missing knowledge . Various methods are explored to facilitate reasoning for missing information . |
| Approach: | They propose a novel knowledge graph reasoning approach that uses a query-related fusion gate unit to model the sequentiality of relation composition and a buffering update mechanism to alleviate lagged entity information propagation. |
| Outcome: | Experimental results show that the proposed approach is superior on both transductive and inductive link prediction tasks. |
Explain by Evidence: An Explainable Memory-based Neural Network for Question Answering (2020.coling-main)
Copied to clipboard
| Challenge: | Interpretability and explainability of deep neural net models are always challenging due to their size and complexity. |
| Approach: | They propose to design an explainable, evidence-based memory network architecture that connects current sample with seen samples and bases its decision on these samples. |
| Outcome: | The proposed model can trace errors to training instances that might have caused errors . the proposed model achieves state-of-the-art performance on two popular datasets . |
Rethinking Complex Neural Network Architectures for Document Classification (N19-1)
Copied to clipboard
| Challenge: | Neural network models for many NLP tasks have grown increasingly complex in recent years . authors of recent papers question the necessity of such architectures and find them quite effective . |
| Approach: | They propose to use regularization techniques borrowed from language modeling to improve model accuracy . they find that a simple biLSTM architecture with appropriate regularization yields competitive results . |
| Outcome: | a simple biLSTM model outperforms the state-of-the-art on four benchmark datasets . authors say that improvements are not real, but are attributed to mundane reasons . |
Topic Memory Networks for Short Text Classification (D18-1)
Copied to clipboard
| Challenge: | Existing classification models for short texts are weak due to data sparsity . |
| Approach: | They propose topic memory networks for short text classification with a novel topic memory mechanism to encode latent topic representations indicative of class labels. |
| Outcome: | The proposed model outperforms state-of-the-art models on short text classification, while generating coherent topics. |
How much complexity does an RNN architecture need to learn syntax-sensitive dependencies? (2020.acl-srw)
Copied to clipboard
| Challenge: | Long-term memory (LSTM) networks are capable of encapsulating long-range dependencies . but simple recurrent networks (SRNs) have been less successful at capturing long-term dependencies and loci of grammatical errors in an unsupervised setting. |
| Approach: | They propose a new architecture that incorporates the decaying nature of neuronal activations and models the excitatory and inhibitory connections in a population of neurons. |
| Outcome: | The proposed architecture shows competitive performance relative to LSTMs on subject-verb agreement, sentence grammaticality, and language modeling tasks. |
HeLa-Mem: Hebbian Learning and Associative Memory for LLM Agents (2026.acl-long)
Copied to clipboard
| Challenge: | Existing memory systems represent conversation history as unstructured embedding vectors, retrieving information through semantic similarity. |
| Approach: | They propose a bio-inspired memory architecture that models memory as a dynamic graph with Hebbian learning dynamics. |
| Outcome: | The proposed architecture leverages both semantic similarity and learned associations . it can be used to build a bio-inspired memory graph with Hebbian learning dynamics . |
Multimodal Neural Graph Memory Networks for Visual Question Answering (2020.acl-main)
Copied to clipboard
| Challenge: | Visual question answering (VQA) is a new challenge for AI. |
| Approach: | They propose a graph neural network architecture based on the recently proposed Graph Network (GN) . they generate visual features and encoded captions for an image to generate two GNs . |
| Outcome: | The proposed model rivals the state-of-the-art models on Visual7W, VQA-v2.0, and CLEVR datasets. |