| Challenge: | Recent LSTM models have been used to model sequential data processing tasks because of their ability to preserve previous information weighted on distance. |
| Approach: | They propose to use a tree-structured tree-based neural network architecture to solve the problem of unbalanced connections between data units inside and outside semantic groups. |
| Outcome: | The proposed model outperforms the state-of-the-art Bidirectional LSTM model on a signal and noise separation task. |
Similar Papers
Investigating Dynamic Routing in Tree-Structured LSTM for Sentiment Analysis (D19-1)
Copied to clipboard
| Challenge: | Existing deep neural network models such as LSTM and tree-LSTM have a bias problem where the words in the tail of a sentence are more heavily emphasized than those in the header. |
| Approach: | They propose a capsule tree-LSTM model that uses dynamic routing to build sentence representations by assigning different weights to nodes according to their contributions to prediction. |
| Outcome: | The proposed model improves on the Stanford Sentiment Treebank and EmoBank datasets. |
How LSTM Encodes Syntax: Exploring Context Vectors and Semi-Quantization on Natural Text (2020.coling-main)
Copied to clipboard
| Challenge: | LSTMs are widely used to capture informative long-term syntactic dependencies, but how they are reflected in their internal vectors for natural text has not been adequately investigated. |
| Approach: | They analyze how syntactic dependencies are reflected in LSTM's internal gates by learning a language model where syntaktic structures are implicitly given. |
| Outcome: | The proposed model can predict whether a word is inside a phrase structure or not from a small number of components of the context-update vector. |
Inducing Grammar from Long Short-Term Memory Networks by Shapley Decomposition (2020.acl-srw)
Copied to clipboard
| Challenge: | a recent study shows that modern neural networks understand sentences implicitly by inducing recursive structures. |
| Approach: | They propose to explicitly induce grammar by tracing the computational process of a long short-term memory network. |
| Outcome: | The proposed model can explicitly induce grammar without external knowledge . tracing the computational process of a long short-term memory network is shown to be effective . |
Recursive Subtree Composition in LSTM-Based Dependency Parsing (N19-1)
Copied to clipboard
| Challenge: | Existing studies show that tree structure modelling on top of sequence modelling is not feasible. |
| Approach: | They propose to recursively compose subtree representations in a biLSTM-based parser to capture subtreas. |
| Outcome: | The proposed model improves performance under ablating the backward LSTM and the forward LS. |
What Part of the Neural Network Does This? Understanding LSTMs by Measuring and Dissecting Neurons (D19-1)
Copied to clipboard
| Challenge: | Biological neural systems consist of a huge number of neurons, and can react to the environment in complicated ways. |
| Approach: | They propose a metric to quantify the sensitivity of neurons to each label and conduct experiments to prove it. |
| Outcome: | The proposed metric is based on a set of experiments that show that dropping an arbitrary neuron significantly degrades the accuracy of the model. |
Graph-to-Tree Neural Networks for Learning Structured Input-Output Translation with Applications to Semantic Parsing and Math Word Problem (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Graph2Tree model encodes graph-structured input and decodes tree-structures output. |
| Approach: | They propose a novel Graph-to-Tree Neural Network consisting of a graph encoder and a hierarchical tree decoder that encodes an augmented graph-structured input and decodes a tree-structure-output. |
| Outcome: | The proposed model outperforms or matches the performance of other state-of-the-art models on two problems, neural semantic parsing and math word problem. |
How much complexity does an RNN architecture need to learn syntax-sensitive dependencies? (2020.acl-srw)
Copied to clipboard
| Challenge: | Long-term memory (LSTM) networks are capable of encapsulating long-range dependencies . but simple recurrent networks (SRNs) have been less successful at capturing long-term dependencies and loci of grammatical errors in an unsupervised setting. |
| Approach: | They propose a new architecture that incorporates the decaying nature of neuronal activations and models the excitatory and inhibitory connections in a population of neurons. |
| Outcome: | The proposed architecture shows competitive performance relative to LSTMs on subject-verb agreement, sentence grammaticality, and language modeling tasks. |
Better Feature Integration for Named Entity Recognition (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to named entity recognition (NER) focus on stacking the LSTM and graph neural networks (GCNs) however, the exact interaction mechanism between the two types of features is not clear and the performance gain is not significant. |
| Approach: | They propose a model that incorporates both types of features with a Synergized-LSTM which captures how the two types of feature interact. |
| Outcome: | The proposed model achieves better performance than previous approaches while requiring fewer parameters. |
DR-BiLSTM: Dependent Reading Bidirectional LSTM for Natural Language Inference (N18-1)
Copied to clipboard
Reza Ghaeini, Sadid A. Hasan, Vivek Datla, Joey Liu, Kathy Lee, Ashequl Qadir, Yuan Ling, Aaditya Prakash, Xiaoli Fern, Oladimeji Farri
| Challenge: | Existing approaches to natural language inference rely on simple reading mechanisms for independent encoding of the premise and hypothesis. |
| Approach: | They propose a novel bidirectional dependent reading network to efficiently model the relationship between a premise and a hypothesis during encoding and inference. |
| Outcome: | The proposed model outperforms existing methods by a considerable margin on the Stanford Natural Language Inference (SNLI) dataset. |
The OSU/Facebook Realizer for SRST 2019: Seq2Seq Inflection and Serialized Tree2Tree Linearization (D19-63)
Copied to clipboard
| Challenge: | Existing linearization methods for shallow surface realization tasks are not available for all languages. |
| Approach: | They propose a system that implements morphological inflection with a baseline linearizer for a shallow surface realization task. |
| Outcome: | The proposed system is competitive across languages, but poor on longer sentences. |