| Challenge: | Existing work exploits the reordering information in neural machine translation . experimental results show that the proposed methods can significantly improve the performance of the transformer translation system. |
| Approach: | They propose a reordering mechanism to learn the re ordering embedding of a word based on contextual information and stack them together with self-attention networks to learn sentence representation for machine translation. |
| Outcome: | The proposed method improves translation performance on English-to-German, NIST Chinese-to English, and WAT Japanese-toEnglish translation tasks. |
Similar Papers
Recurrent Positional Embedding for Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Existing translation systems that use positional embeddings only encode static order dependencies based on discrete numerical information, which may hinder the improvement of translation capacity. |
| Approach: | They propose a recurrent positional embedding approach based on word vectors that are learned by a neural network and integrated into existing multi-head self-attention models. |
| Outcome: | The proposed approach improves translation performance over the state-of-the-art Transformer baseline in English-to-German and NIST Chinese-to English translation tasks. |
Assessing Non-autoregressive Alignment in Neural Machine Translation via Word Reordering (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing non-autoregressive neural machine translation models that implicitly model dependencies are sub-optimal in handling word order errors. |
| Approach: | They propose to learn a non-autoregressive language model that can be combined with Viterbi decoding to achieve better reordering performance. |
| Outcome: | The proposed model outperforms state-of-the-art reordering mechanisms under different word permutation settings with a 2-27 BLEU improvement, suggesting high potential for word alignment in NAT. |
Improving Transformer Models by Reordering their Sublayers (2020.acl-main)
Copied to clipboard
| Challenge: | a sandwich transformer pattern is a new approach to multilayer transformers that can be used for different tasks. |
| Approach: | They propose a transformer ordering pattern that reorders sublayers in a sandwich transformer pattern . they generate random transformer models and train them with the language modeling objective . |
| Outcome: | The proposed pattern improves perplexity on multiple word-level and character-level language modeling benchmarks at no cost in parameters, memory, or training time. |
Discriminative Reranking for Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | reranking models allow the integration of rich features to select a better output hypothesis within an n-best list or lattice. |
| Approach: | They use discriminative reranking to train a large transformer architecture to train an ranked list of hypotheses. |
| Outcome: | Experiments on four WMT directions show that discriminative reranking improves translation quality. |
Neural Syntactic Preordering for Controlled Paraphrase Generation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to paraphrasing natural language sentences are limited by the complexity of the task. |
| Approach: | They propose a framework for paraphrasing natural language sentences that uses syntactic transformations to softly "reorder" the source sentence and their proposed system is evaluated automatically and by humans . |
| Outcome: | The proposed model retains the quality of the baseline approaches while giving a substantial increase in the diversity of the generated paraphrases. |
Exploiting Pre-Ordering for Neural Machine Translation (L18-1)
Copied to clipboard
| Challenge: | Existing studies have shown that Neural Machine Translation suffers from the problems that some source words are mistakenly translated for multiple times . |
| Approach: | They propose a pre-ordering approach to solve the under-translation problem by pre-ordnanced source sentences and position embedding to enhance monotone translation. |
| Outcome: | The proposed method significantly improves translation quality by 2.43 BLEU points on Chinese-to-English translation. |
Assessing the Ability of Self-Attention Networks to Learn Word Order (P19-1)
Copied to clipboard
| Challenge: | Existing studies have attributed SAN to being weak at learning positional information for sequence modeling due to lack of recurrence structure. |
| Approach: | They propose a word reordering detection task to quantify how well word order information is learned by SAN and RNN. |
| Outcome: | The proposed task quantifies how well word order information learned by SAN and RNN is learned. |
Enhancing Machine Translation with Dependency-Aware Self-Attention (2020.acl-main)
Copied to clipboard
| Challenge: | Currently, most neural machine translation models rely on pairs of parallel sentences, assuming syntactic information is automatically learned by an attention mechanism. |
| Approach: | They propose a parameter-free, dependency-aware self-attention mechanism that integrates syntactic knowledge into a Transformer model and propose 'a parameter free approach' they also propose - a novel mechanism that improves translation quality for long sentences and in low-resource scenarios. |
| Outcome: | The proposed approach improves translation quality on English-German and English-Turkish translation tasks and in low-resource scenarios. |
A Discriminative Neural Model for Cross-Lingual Word Alignment (D19-1)
Copied to clipboard
| Challenge: | a novel word alignment model for machine translation has been developed for a number of languages . explicit word-to-word alignments have largely been lost in neural MT systems . |
| Approach: | They propose a discriminative word alignment model which integrates into a Transformer-based machine translation model. |
| Outcome: | The proposed model performs better on Chinese and Arabic alignments than standard models. |
Context-Aware Neural Machine Translation Decoding (D19-65)
Copied to clipboard
| Challenge: | Existing approaches to enhance neural machine translation systems to take into account document-level information make the training process slower or require document- level annotated data. |
| Approach: | They propose a decoding architecture that fuses the semantic space language model and a neural translation model. |
| Outcome: | The proposed approach improves translation quality for English–Spanish using BLEU and METEOR. |