| Challenge: | Contextual sequence mapping is one of the fundamental problems in Natural Language Processing (NLP). |
| Approach: | They propose a new family of Recurrent Neural Networks that address contextual sequence mapping . they propose to use contextual signals to control the flow of information . |
| Outcome: | The proposed architecture outperforms existing methods on dialog problem and language model . the proposed architectures are based on a novel family of recurrent neural networks . |
Similar Papers
Convolutional Neural Networks with Recurrent Neural Filters (D18-1)
Copied to clipboard
| Challenge: | Convolutional neural networks (CNNs) use recurrent neural networks as convolution filters to capture language compositionality and long-term dependencies. |
| Approach: | They propose to use recurrent neural networks (RNNs) as convolution filters to capture language compositionality and long-term dependencies. |
| Outcome: | The proposed convolutional neural networks achieve state-of-the-art on two sentences and the Stanford Sentiment Treebank. |
Advancing Regular Language Reasoning in Linear Recurrent Neural Networks (2024.naacl-short)
Copied to clipboard
| Challenge: | Existing linear recurrent neural networks have been used for natural language and long-range modeling for decades. |
| Approach: | They propose a linear recurrent neural network with a block-diagonal transition matrix and a transition matrix for LRNNs. |
| Outcome: | The proposed model is the only one capable of performing length extrapolation on regular language tasks such as Sum, Even Pair, and Modular Arithmetic. |
A Hybrid Neural Network Model for Commonsense Reasoning (D19-60)
Copied to clipboard
| Challenge: | a hybrid neural network (HNN) model for commonsense reasoning is proposed . it combines language models and semantic similarity models to achieve new state-of-the-art results . |
| Approach: | They propose a hybrid neural network model for commonsense reasoning . it combines a masked language model and a semantic similarity model . |
| Outcome: | The proposed model outperforms the WNLI, WSC and PDP60 benchmarks on three commonsense reasoning tasks. |
Recurrent Neural Networks with Mixed Hierarchical Structures and EM Algorithm for Natural Language Processing (2022.lrec-1)
Copied to clipboard
| Challenge: | A variety of hierarchical RNN models have been proposed to incorporate hierarchically-based hierarchic information in modeling languages in the literature. |
| Approach: | They propose a latent indicator layer approach to identify and learn hierarchical information and develop an EM algorithm to handle the latent indicators layer in training. |
| Outcome: | The proposed approach outperforms other RNN-based models in document classification tasks. |
Efficient Sequence Learning with Group Recurrent Networks (N18-1)
Copied to clipboard
| Challenge: | Recurrent neural networks have achieved state-of-the-art results in many artificial intelligence tasks, such as language modeling, neural machine translation and speech recognition. |
| Approach: | They propose an efficient architecture to improve the efficiency of such RNN model training by adopting the group strategy for recurrent layers while exploiting the representation rearrangement strategy between layers as well as time steps. |
| Outcome: | The proposed architecture achieves comparable or better accuracy compared with baselines, with a much smaller number of parameters and at a lower computational cost. |
Searching for Effective Neural Extractive Summarization: What Works and What’s Next (P19-1)
Copied to clipboard
| Challenge: | Recent years have seen success in the use of deep neural networks on text summarization, but there is no clear understanding of why they perform so well or how they might be improved. |
| Approach: | They propose to use different types of model architectures to improve extractive summarization systems. |
| Outcome: | The proposed framework achieves state-of-the-art on CNN/DailyMail by a large margin based on observations and analysis. |
Direct Output Connection for a High-Rank Language Model (D18-1)
Copied to clipboard
| Challenge: | Neural network language models have played a central role in recent natural language processing advances. |
| Approach: | They propose a recurrent neural network language model that combines probability distributions computed from a final RNN layer and also middle layers. |
| Outcome: | The proposed language model achieves the best score on the Penn Treebank and WikiText-2 benchmark datasets. |
Marrying Up Regular Expressions with Neural Networks: A Case Study for Spoken Language Understanding (P18-1)
Copied to clipboard
| Challenge: | Experimental results show that the combination of regular expressions and NNs improves learning effectiveness when a small number of training examples are available. |
| Approach: | They propose to combine a neural network (NN) with regular expressions (RE) to improve supervised learning for NLP by exploiting the rich expressiveness of REs at different levels within a NN. |
| Outcome: | The proposed approach significantly improves learning effectiveness when a small number of training examples are available. |
The Importance of Being Recurrent for Modeling Hierarchical Structure (D18-1)
Copied to clipboard
| Challenge: | Recent work shows that recurrent neural networks can implicitly capture hierarchical information when trained to solve common natural language processing tasks. |
| Approach: | They propose a convolutional sequence-to-sequence model that exploits hierarchical information implicitly. |
| Outcome: | The proposed model is recurrent and non-recurrent, and it can model hierarchical structure implicitly. |
Priorless Recurrent Networks Learn Curiously (2020.coling-main)
Copied to clipboard
| Challenge: | a recent study shows domain-general recurrent neural networks reproduce human language behaviours . a lack of a unified concept of number agreement between these processes is a limitation of the model . |
| Approach: | They propose to use domain-general recurrent neural networks without explicit linguistic inductive biases to reproduce human language behaviours. |
| Outcome: | The proposed model can learn number agreement within unnatural sentences, the authors show . they show that the model has an effective understanding of singular versus plural for individual sentences . |