| Challenge: | Convolutional neural networks (CNNs) use recurrent neural networks as convolution filters to capture language compositionality and long-term dependencies. |
| Approach: | They propose to use recurrent neural networks (RNNs) as convolution filters to capture language compositionality and long-term dependencies. |
| Outcome: | The proposed convolutional neural networks achieve state-of-the-art on two sentences and the Stanford Sentiment Treebank. |
Similar Papers
Adaptive Convolution for Text Classification (N19-1)
Copied to clipboard
| Challenge: | Existing convolutional neural networks (CNNs) use sparse representations of text, such as bag-of-words. |
| Approach: | They propose an adaptive convolution for text classification to give flexibility to convolutional neural networks (CNNs) they attach filter-generating networks to convevolution blocks in existing CNNs . |
| Outcome: | The proposed convolution improves performance in seven benchmark datasets by 2.6 percentage points . the proposed conversions can be likened to players of the twenty questions . |
Searching for Effective Neural Extractive Summarization: What Works and What’s Next (P19-1)
Copied to clipboard
| Challenge: | Recent years have seen success in the use of deep neural networks on text summarization, but there is no clear understanding of why they perform so well or how they might be improved. |
| Approach: | They propose to use different types of model architectures to improve extractive summarization systems. |
| Outcome: | The proposed framework achieves state-of-the-art on CNN/DailyMail by a large margin based on observations and analysis. |
Recurrent Neural Networks as Weighted Language Recognizers (N18-1)
Copied to clipboard
| Challenge: | Recent experiments show that RNNs outperform other methods in assigning high probability to held-out English text. |
| Approach: | They focus on the single-layer, ReLU-activation, rational-weight RNNs with softmax . they show that most problems for such RNN are undecidable . |
| Outcome: | The proposed model outperforms other methods in assigning high probability to held-out English text. |
Sequential Learning of Convolutional Features for Effective Text Classification (D19-1)
Copied to clipboard
| Challenge: | Existing models for text classification have largely ignored convolution filters and max pooling . text classification is one of the major applications of natural language processing . |
| Approach: | They propose a convolutional attentive recurrent network model which uses convolution filters and max pooling to improve text classification. |
| Outcome: | The proposed model outperforms existing convolutional models on text classification tasks. |
Restricted Recurrent Neural Tensor Networks: Exploiting Word Frequency and Compositionality (P18-2)
Copied to clipboard
| Challenge: | Recurrent neural tensor networks (RNNs) increase capacity by augmenting the size of the hidden layer, with significant increase in computational cost. |
| Approach: | They propose restricted recurrent neural tensor networks (r-RNTNs) which reserve distinct hidden layer weights for frequent vocabulary words while sharing a single set of weights . |
| Outcome: | The proposed model outperforms unrestricted RNTNs using only a small fraction of the parameters of unrestrained RNNNs. |
The Importance of Being Recurrent for Modeling Hierarchical Structure (D18-1)
Copied to clipboard
| Challenge: | Recent work shows that recurrent neural networks can implicitly capture hierarchical information when trained to solve common natural language processing tasks. |
| Approach: | They propose a convolutional sequence-to-sequence model that exploits hierarchical information implicitly. |
| Outcome: | The proposed model is recurrent and non-recurrent, and it can model hierarchical structure implicitly. |
Implicit n-grams Induced by Recurrence (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent studies show that self-attention based models have limitations on modeling sequential transformations. |
| Approach: | They propose to extract some explainable features from trained RNNs that are reminiscent of classical n-grams features. |
| Outcome: | The proposed models can model interesting linguistic phenomena such as negation and intensification. |
On Efficiently Representing Regular Languages as RNNs (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent work by Hewitt et al. (2020) provides an interpretation of the empirical success of recurrent neural networks (RNNs) as language models (LMs). |
| Approach: | They generalize their construction and show that RNNs can efficiently represent a larger class of LMs than previously claimed. |
| Outcome: | The results suggest that RNNs can represent a larger class of LMs than previously claimed . |
Learning Context-Sensitive Convolutional Filters for Text Processing (D18-1)
Copied to clipboard
| Challenge: | Convolutional neural networks (CNNs) are a popular building block for natural language processing . despite their success, most existing CNN models share the same learned set of filters for all input sentences. |
| Approach: | They propose to use a meta network to learn context-sensitive convolutional filters for text processing by using a bidirectional filter generation mechanism. |
| Outcome: | The proposed framework outperforms standard and attention-based CNN models on four different tasks. |
Why Self-Attention? A Targeted Evaluation of Neural Machine Translation Architectures (D18-1)
Copied to clipboard
| Challenge: | Recent studies show that non-recurrent architectures outperform RNNs in neural machine translation. |
| Approach: | They hypothesize that CNNs and self-attentional networks could extract semantic features from source text. |
| Outcome: | The proposed architectures outperform RNNs on two tasks: subject-verb agreement and word sense disambiguation. |