Papers with IWSLT
Learning Language Specific Sub-network for Multilingual Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Multilingual neural machine translation models suffer from performance degradation when learning multiple languages. |
| Approach: | They propose to use LaSS to jointly train a single unified multilingual MT model. |
| Outcome: | The proposed model gains on 36 language pairs by up to 1.2 BLEU and zero-shot translation with 8.3 BLUE on 30 language pairs. |
Gradient-based Gradual Pruning for Language-Specific Multilingual Neural Machine Translation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual neural machine translation suffers from performance degradation in high-resource languages compared to bilingual counterparts. |
| Approach: | They propose a gradient-based gradual pruning technique for multilingual neural machine translation that allows for partial parameter sharing across language pairs to alleviate interference. |
| Outcome: | The proposed approach yields a notable performance gain on IWSLT and WMT datasets. |
Diverse Pretrained Context Encodings Improve Document Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models for sentence-level sequence-to-sequence translations do not use extra-sentential information. |
| Approach: | They propose a sentence-level sequence-to-sequence transformer with multiple pre-trained context signals. |
| Outcome: | The proposed model outperforms existing models on Chinese-English and English-German tasks. |
Exploring the Impact of Layer Normalization for Zero-shot Neural Machine Translation (2023.acl-short)
Copied to clipboard
| Challenge: | Recent studies have shown that layer normalization (LayerNorm) overfits training data and therefore has low generalizability for ZST. |
| Approach: | They propose to use the Transformer architecture to set the default layer normalization setting for zero-shot translation (ZST) they also propose to set LayerNorm after residual connections to outperform PreNorm by 12.3 BLEU points. |
| Outcome: | The proposed model outperforms the current model by 12.3 BLEU points on 54 directions on OPUS, IWSLT, and Europarl datasets. |
Non-Parametric Adaptation for Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | Neural Networks trained with gradient descent are susceptible to catastrophic forgetting due to parameter shift during the training process. |
| Approach: | They propose a semi-parametric approach that relies on local phrase level similarities to retrieve neighboring phrases that are useful for translation even when overall sentence similarity is low. |
| Outcome: | The proposed approach performs well on a heterogeneous dataset with WMT, IWSLT, JRC-Acquis and OpenSubtitles. |
Counter-Interference Adapter for Multilingual Machine Translation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to multilingual machine translation suffer from performance degradation, resulting in a single model being inferior to separately trained bilingual models on resource-rich languages. |
| Approach: | They propose a transformer-based model with a small parameter overhead for multilingual machine translation that outperforms strong multilingual baselines on 64 of 66 language directions. |
| Outcome: | The proposed model outperforms strong multilingual baselines on 64 of 66 language directions, 42 of which have above 0.5 BLEU improvement. |
Collective Wisdom: Improving Low-resource Neural Machine Translation using Adaptive Knowledge Distillation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to train high-quality NMT models in bilingually low-resource scenarios are limited by the scarcity of parallel sentence-pairs. |
| Approach: | They propose to distill the knowledge of teacher models to a single student model by using knowledge distillation. |
| Outcome: | The proposed approach achieves up to +0.9 BLEU score improvements compared to strong baselines. |
Does Multi-Encoder Help? A Case Study on Context-Aware Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | In encoder-decoder neural models, multiple encoders are used to represent contextual information in addition to the individual sentence. |
| Approach: | They propose to use multiple context encoders to encode the individual sentences in document-level neural machine translation (NMT) They propose a noisy dropout setup and a single-encoder approach to encode context sentences. |
| Outcome: | The proposed approach encodes the context and the current sentence without contexts. |
Synchronously Generating Two Languages with Interactive Decoding (D19-1)
Copied to clipboard
| Challenge: | Experimental results show that multilingual NMT models handle multiple language pairs in one model. |
| Approach: | They propose an interactive approach to translate a source language into two different languages simultaneously and interactively. |
| Outcome: | The proposed approach improves on IWSLT and WMT datasets. |
Importance-based Neuron Allocation for Multilingual Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to multilingual neural machine translation tend to preserve general knowledge, but ignore language-specific knowledge. |
| Approach: | They propose to divide model neurons into general and language-specific parts based on their importance across languages. |
| Outcome: | The proposed model can preserve general knowledge but ignore language-specific knowledge on several languages, and is universal and cost-effective. |
Modeling Context With Linear Attention for Scalable Document-Level Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Document-level machine translation models lack quadratic complexity in the sequence length due to their attention layers. |
| Approach: | They evaluate a recent linear attention model with a sentential gate to promote a recency inductive bias and compare it to open-source document translation. |
| Outcome: | The proposed model significantly improves translation quality on IWSLT 2015 and OpenSubtitles 2018 with similar or better BLEU scores. |