Simplifying Neural Machine Translation with Addition-Subtraction Twin-Gated Recurrent Networks (D18-1)
Copied to clipboard
| Challenge: | Existing gated recurrent networks have a vanishing gradient, allowing for more matrix transformations and less transparent functions. |
| Approach: | They propose an additionsubtraction twin-gated recurrent network (ATR) to simplify neural machine translation. |
| Outcome: | The proposed system is more transparent than LSTM/GRU due to the simplification. |
Similar Papers
Self-Attentive Residual Decoder for Neural Machine Translation (N18-1)
Copied to clipboard
| Challenge: | Neural sequence-to-sequence networks with attention have been used for machine translation . however, the target-side context is limited and the model lacks the ability to capture non-syntactic dependencies among words. |
| Approach: | They propose a sequence-to-sequence network with attention that captures contextual information at each time-step prediction through an attention mechanism. |
| Outcome: | The proposed model outperforms a neural MT baseline and memory and self-attention network on three language pairs. |
A Lightweight Recurrent Network for Sequence Modeling (P19-1)
Copied to clipboard
| Challenge: | Recent studies show that recurrent networks suffer from severe computational inefficiency due to weak parallelization. |
| Approach: | They propose a lightweight recurrent network (LRN) that uses input and forget gates to handle long-range dependencies and gradient vanishing and explosion. |
| Outcome: | The proposed recurrent network yields the best running efficiency on six NLP tasks. |
Learning to Rewrite for Non-Autoregressive Neural Machine Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing non-autoregressive neural machine translations have poor inference speed but weak recognition of erroneous translation pieces. |
| Approach: | They propose an architecture to explicitly learn to rewrite the erroneous translation pieces. |
| Outcome: | The proposed architecture can achieve better performance while significantly reducing decoding time. |
Fusing Recency into Neural Machine Translation with an Inter-Sentence Gate Model (C18-1)
Copied to clipboard
| Challenge: | Neural machine translation systems translate one sentence at a time, ignoring inter-sentence information. |
| Approach: | They propose an inter-sentence gate model that uses the same encoder to encode two adjacent sentences . it captures the connection between sentences and fuses recency from neighboring sentences a model proposes . |
| Outcome: | The proposed model improves on NIST Chinese-English translation tasks . it captures the connection between sentences and fuses recency from neighboring sentences . |
Refining Source Representations with Relation Networks for Neural Machine Translation (C18-1)
Copied to clipboard
| Challenge: | Existing neural machine translation frameworks that forget distant information and disregard relationship between source and target words are not effective. |
| Approach: | They propose to use relation networks to learn better representations of the source . they propose to associate source words with each other to help retain their relationships . |
| Outcome: | Experiments show that the proposed approach outperforms the encoder-decoder framework on several datasets. |
Graph-to-Sequence Learning using Gated Graph Neural Networks (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to graph-to-sequence learning ignore the full graph structure, discarding key information. |
| Approach: | They propose a graph-to-sequence learning model that encodes the full graph structure and an input transformation that allows nodes and edges to have their own hidden representations. |
| Outcome: | The proposed model outperforms baselines in generation from AMR graphs and syntax-based neural machine translation while retaining the full graph structure. |
Advancing Regular Language Reasoning in Linear Recurrent Neural Networks (2024.naacl-short)
Copied to clipboard
| Challenge: | Existing linear recurrent neural networks have been used for natural language and long-range modeling for decades. |
| Approach: | They propose a linear recurrent neural network with a block-diagonal transition matrix and a transition matrix for LRNNs. |
| Outcome: | The proposed model is the only one capable of performing length extrapolation on regular language tasks such as Sum, Even Pair, and Modular Arithmetic. |
Tailoring Neural Architectures for Translating from Morphologically Rich Languages (C18-1)
Copied to clipboard
| Challenge: | A morphologically complex word is a hierarchical constituent with meaning-preserving subunits, so word-based models which rely on surface forms might not be powerful enough to translate such structures. |
| Approach: | They propose a neural architecture which is designed to deal with morphological complexities on the source side and redesign the decoder accordingly to benefit from such information. |
| Outcome: | The proposed model outperforms existing subword- and character-based architectures and showed significant improvements on translating from German, Russian, and Turkish into English. |
Modeling Recurrence for Transformer (N19-1)
Copied to clipboard
| Challenge: | Existing studies show that the lack of recurrence modeling hinders the development of a translation model. |
| Approach: | They propose to model recurrence for Transformer with an additional recurrent encoder. |
| Outcome: | The proposed model outperforms the deep model on EnglishGerman and ChineseEnglish translation tasks. |
Improving Non-Autoregressive Neural Machine Translation via Modeling Localness (2022.coling-1)
Copied to clipboard
| Challenge: | Existing non-autoregressive neural machine translation models suffer from poor localization quality due to sequential dependencies within the target sentence. |
| Approach: | They propose to introduce local information into NAT models by explicitly introducing local information about surrounding words into the encoder and decoder sides to achieve localness-aware representations. |
| Outcome: | The proposed method can achieve significant improvements over strong NAT baselines. |