Papers with NMT
Domain Adaptation in Neural Machine Translation using a Qualia-Enriched FrameNet (2022.lrec-1)
Copied to clipboard
| Challenge: | Neural models have been advancing in a myriad of tasks, but there is a lack of large training data. |
| Approach: | They propose a method for domain adaptation of Neural Machine Translation systems using a multilingual FrameNet enriched with qualia relations as an external knowledge base. |
| Outcome: | The proposed system outperforms the state-of-the-art commercial system in an experiment . the proposed system substitutes domain-specific terms in the source language by their adequate translation in the target language. |
Zero-Shot Cross-Lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on improving cross-lingual transferability of NMT model is under-explored. |
| Approach: | They propose a model that leverages a multilingual pretrained encoder to improve cross-lingual transferability. |
| Outcome: | The proposed model outperforms mBART and m2m-100 on a zero-shot cross-lingual transfer task. |
Multilingual Neural Machine Translation (2020.coling-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we will cover the latest advances in NMT to enhance low-resource translation. |
| Approach: | They will cover the latest advances in NMT approaches that leverage multilingualism . they will focus on topics such as language divergence, transfer learning and pivoting . |
| Outcome: | This tutorial will cover the latest advances in NMT to enhance low-resource translation models. |
Priming Ancient Korean Neural Machine Translation (2022.lrec-1)
Copied to clipboard
| Challenge: | Recent studies have focused on the restoration and translation of historical languages. |
| Approach: | They propose to use two different stimuli to priming ancient-Korean NMT . they confirm the possibility of developing a human-centric model based on cognitive science . |
| Outcome: | The proposed model can be used to translate historical Korean documents using neural machine translation. |
PENELOPIE: Enabling Open Information Extraction for the Greek Language through Machine Translation (2021.eacl-srw)
Copied to clipboard
| Challenge: | a new method for information extraction from Greek corpora is being developed for low-resource languages. |
| Approach: | They propose a methodology that aims at bridging the gap between high and low-resource languages in the context of Open Information Extraction. |
| Outcome: | The proposed method outperforms the current state-of-the-art for the Greek language on benchmark datasets. |
Salute the Classic: Revisiting Challenges of Machine Translation in the Age of Large Language Models (2025.tacl-1)
Copied to clipboard
| Challenge: | a recent study revisits six core challenges that have influenced the evolution of Neural Machine Translation (NMT) domain mismatch, amount of parallel data, rare word prediction, translation of long sentences and sub-optimal beam search remain challenges in LLMs. |
| Approach: | They revisit core challenges that have acted as benchmarks for progress in NMT . they propose to revisit these challenges and offer insights into their relevance . |
| Outcome: | The proposed models significantly improve translation of sentences containing approximately 80 words, even translating documents up to 512 words. |
Pre-training Methods for Neural Machine Translation (2021.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial provides a comprehensive guide to make the most of pre-training for neural machine translation. |
| Approach: | This tutorial provides a comprehensive guide to make the most of pre-training for neural machine translation. |
| Outcome: | This tutorial explains how to make the most of pre-training for neural machine translation. |
Unsupervised Neural Machine Translation with Weight Sharing (P18-1)
Copied to clipboard
| Challenge: | Unsupervised neural machine translation (NMT) is a new approach for machine translation . the model uses only one shared encoder to map pairs of sentences from different languages to a shared-latent space . |
| Approach: | They propose an unsupervised approach which trains the model without labeling data . they propose two independent encoders but share some partial weights to extract high-level representations of input sentences. |
| Outcome: | The proposed approach achieves significant improvements on English-German, English-French and Chinese-to-English translation tasks. |
Supervised and Unsupervised Machine Translation for Myanmar-English and Khmer-English (D19-52)
Copied to clipboard
Benjamin Marie, Hour Kaing, Aye Myat Mon, Chenchen Ding, Atsushi Fujita, Masao Utiyama, Eiichiro Sumita
| Challenge: | Using cleaned and normalized noisy monolingual data, supervised neural and statistical machine translation systems performed among the best for the four translation directions. |
| Approach: | They present supervised and unsupervised machine translation systems for the WAT2019 Myanmar-English and Khmer-English translation tasks. |
| Outcome: | The proposed systems performed among the best for the four translation directions. |
Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasing (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing metrics for machine translation evaluation are causing the correlation between human judgments and automatic metrics to break down. |
| Approach: | They propose to train a multilingual NMT system to score machine translation output with a sequence-to-sequence paraphraser, conditioned on a human reference. |
| Outcome: | The proposed model outperforms or statistically ties with all prior metrics on the WMT 2019 segment-level shared metrics task in all languages (excluding Gujarati where the model had no training data). |
Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages (2022.tacl-1)
Copied to clipboard
Gowtham Ramesh, Sumanth Doddapaneni, Aravinth Bheemaraj, Mayank Jobanputra, Raghavan AK, Ajitesh Sharma, Sujit Sahoo, Harshita Diddee, Mahalakshmi J, Divyanshu Kakwani, Navneet Kumar, Aswin Pradeep, Srihari Nagaraj, Kumar Deepak, Vivek Raghavan, Anoop Kunchukuttan, Pratyush Kumar, Mitesh Shantadevi Khapra
| Challenge: | We present Samanantar, the largest publicly available parallel corpora collection for Indic languages . based on existing corporative, there has been limited benefit for resource-poor languages despite the lack of parallel corporals and monolingual corporata. |
| Approach: | They compile 12.4 million sentence pairs from existing corpora and mine 37.4 million from the Web. |
| Outcome: | The proposed model outperforms existing models and benchmarks on public datasets. |
Empirical Evaluation of Active Learning Techniques for Neural MT (D19-61)
Copied to clipboard
| Challenge: | Several active learning (AL) algorithms for machine translation (MT) have been well-studied for phrase-based MT. |
| Approach: | They propose to use a phrase-based algorithm to compare different AL methods in a simulated AL framework to demonstrate how unsupervised pre-training and paraphrastic embedding can be used to improve existing AL methods. |
| Outcome: | The proposed method outperforms existing methods in the context of phrase-based MT and is based on a simulated phrase-driven dataset. |
Subset Retrieval Nearest Neighbor Machine Translation (2023.acl-long)
Copied to clipboard
| Challenge: | k-nearest-neighbor machine translation (kNN-MT) is a new approach to improve NMT performance without additional training. |
| Approach: | They propose a method that integrates example-search into the decoding algorithm to improve neighbor token retrieval. |
| Outcome: | The proposed method achieves a speed-up of up to 132.2 times and an improvement in BLEU score of up 1.6 compared with kNN-MT in the WMT’19 translation task and the domain adaptation tasks in De-En and En-Ja. |
SuperNMT: Neural Machine Translation with Semantic Supersenses and Syntactic Supertags (P18-3)
Copied to clipboard
| Challenge: | Neural Machine Translation models have become the state-of-the-art in the field of machine translation. |
| Approach: | They incorporate semantic supersensetags and syntactic supertag features into EN–FR and EN–DE factored NMT systems and show that they improve model training. |
| Outcome: | The proposed model training improves on EN–FR and EN–DE factored NMT systems. |
Introducing the CURLICAT Corpora: Seven-language Domain Specific Annotated Corpora from Curated Sources (2022.lrec-1)
Copied to clipboard
Tamás Váradi, Bence Nyéki, Svetla Koeva, Marko Tadić, Vanja Štefanec, Maciej Ogrodniczuk, Bartłomiej Nitoń, Piotr Pęzik, Verginica Barbu Mititelu, Elena Irimia, Maria Mitrofan, Dan Tufiș, Radovan Garabík, Simon Krek, Andraž Repar
| Challenge: | The CURLICAT CEF Telecom project aims to collect and deeply annotate a set of large corpora from selected domains. |
| Approach: | They present the results of the CURLICAT CEF Telecom project . they propose to collect and deeply annotate a set of large corpora from selected domains . |
| Outcome: | The CURLICAT CEF Telecom project provides a set of large corpora from selected domains . the corporatized corporates are tokenized, lemmatized and morphologically analysed . |
DP-NMT: Scalable Differentially Private Machine Translation (2024.eacl-demo)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is a popular text generation task, yet there is nagging data privacy concerns. |
| Approach: | They propose an open-source framework for a privacy-preserving NMT with DP-SGD. |
| Outcome: | The proposed framework is open-source and open to the public . it combines models, datasets, and evaluation metrics to demonstrate its effectiveness. |
Multimodal Machine Translation with Embedding Prediction (N19-3)
Copied to clipboard
| Challenge: | Pretrained word embeddings improve multimodal machine translation of low-resource domains due to a shortage of training data. |
| Approach: | They propose to combine pretrained word embeddings with search-based approaches to improve NMT of low-resource domains to better translate rare words. |
| Outcome: | The proposed approach improves translation performance by 1.24 METEOR and 2.49 BLEU and achieves 7.67 F-score. |
SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing (D18-2)
Copied to clipboard
| Challenge: | Existing subword segmentation tools assume input is pre-tokenized into word sequences, but SentencePiece can train subword models directly from raw sentences. |
| Approach: | They propose a language-independent subword tokenizer and detokenizer for Neural-based text processing. |
| Outcome: | The proposed system achieves comparable accuracy to training from raw sentences. |
Low-resource neural machine translation with morphological modeling (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for character-based and sub-word tokenization are limited to the surface forms of the words. |
| Approach: | They propose a framework-solution for modeling complex morphology in low-resource settings using a transformer architecture and beam search-based decoder. |
| Outcome: | The proposed model improves translation performance on Kinyarwanda English translation using public-domain parallel text. |
Training with Adversaries to Improve Faithfulness of Attention in Neural Machine Translation (2020.aacl-srw)
Copied to clipboard
| Challenge: | Existing approaches to measure faithfulness of neural machine translation models are based on stress tests and a novel objective that rewards faithful behaviour by the model through probability divergence. |
| Approach: | They propose a measure of faithfulness for neural machine translation models based on stress tests and measuring faithfulness based upon how often the model output changes. |
| Outcome: | The proposed objective increases faithfulness without reducing translation quality and can even improve translation quality in some cases. |
A Study of Non-autoregressive Model for Sequence Generation (2020.acl-main)
Copied to clipboard
| Challenge: | Non-autoregressive (NAR) models generate all tokens in parallel, resulting in faster generation speed compared to autoregressive models. |
| Approach: | They propose to use knowledge distillation and source-target alignment to bridge the gap between NAR and autoregressive models in various tasks. |
| Outcome: | The proposed techniques can speed up NAR models in some tasks but not all . the proposed techniques reduce target token dependency while allowing for faster inference . |
Gender bias amplification during Speed-Quality optimization in Neural Machine Translation (2021.acl-short)
Copied to clipboard
| Challenge: | et al., 2002) show that gendered noun translation performance degrades faster than BLEU. |
| Approach: | They propose to use greedy search, quantization, AANs and shallow decoders to speed up decoding . they find minimal degradation of BLEU, but gendered noun translation degrades faster . |
| Outcome: | The proposed model degrades gendered noun translation performance faster than other models. |
LTRC-MT Simple & Effective Hindi-English Neural Machine Translation Systems at WAT 2019 (D19-52)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) is a promising approach for low resource languages. |
| Approach: | They propose to use both Recurrent Neural Networks & Transformer architectures to train NMT models. |
| Outcome: | The proposed model outperforms Statistical Machine Translation (SMT) techniques on a low resource Hindi-English language pair. |
Neural Machine Translation for Low Resource Languages using Bilingual Lexicon Induced from Comparable Corpora (N18-4)
Copied to clipboard
| Challenge: | Recent crowdsourcing efforts and workshops on machine translation have resulted in small amounts of parallel texts for building viable machine translation systems for low resource pairs. |
| Approach: | They propose to use an end-to-end Siamese bidirectional recurrent neural network to extract parallel sentences from Wikipedia to improve BLEU scores on both NMT and phrase-based SMT systems. |
| Outcome: | The proposed approach improves BLEU scores on both NMT and phrase-based SMT systems for the low-resource language pairs English–Hindi and English–Tamil when compared to training exclusively on the limited bilingual corpora. |
FreeTransfer-X: Safe and Label-Free Cross-Lingual Transfer from Off-the-Shelf Models (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing work on cross-lingual transfer has not studied how to leverage knowledge of rich-resource languages without labels. |
| Approach: | They propose a 2-step knowledge distillation framework to achieve knowledge transfer from off-the-shelf models in rich-resource languages. |
| Outcome: | The proposed method reduces annotation cost and protects private labels. |
Unsupervised Pivot Translation for Distant Languages (P19-1)
Copied to clipboard
| Challenge: | Unsupervised neural machine translation (NMT) is a popular method for transferring information between languages. |
| Approach: | They propose an unsupervised pivot translation method which translates a language to a distant language through multiple hops. |
| Outcome: | The proposed method improves translation on 20 languages and 294 distant languages on 20 different languages and language pairs. |
Parallel sentences mining with transfer learning in an unsupervised setting (2021.naacl-srw)
Copied to clipboard
| Challenge: | Existing methods to mine parallel sentences in low-resource environments are not suitable for many low-level language pairs. |
| Approach: | They propose an approach based on transfer learning to mine parallel sentences in an unsupervised setting using bilingual corpora of low-resource language pairs. |
| Outcome: | The proposed model improves the performance of mined parallel sentences at two real-world low-resource language pairs compared with previous methods. |
Adaptively Scheduled Multitask Learning: The Case of Low-Resource Neural Machine Translation (D19-56)
Copied to clipboard
| Challenge: | Neural Machine Translation suffers from the lack of bilingual data in low-resource scenarios. |
| Approach: | They propose to inject inductive biases into Neural Machine Translation (NMT) using auxiliary syntactic and semantic tasks. |
| Outcome: | The proposed approach improves translation quality by reweighing training data of main and auxiliary tasks based on their contributions to generalisability of main task. |
Cultural and Geographical Influences on Image Translatability of Words across Languages (2021.naacl-main)
Copied to clipboard
| Challenge: | Neural machine translation models produce poor translations when there are few/no parallel sentences to train the models. |
| Approach: | They define image translatability as the translability of words as images associated with words in different languages that have a high degree of visual similarity. |
| Outcome: | The proposed model improves upon text-only models only marginally. |
An Effective Approach to Unsupervised Machine Translation (P19-1)
Copied to clipboard
| Challenge: | a recent research line has managed to train both unsupervised and unsupervised machine translation systems using monolingual corpora only. |
| Approach: | They propose to use monolingual corpora to train both unsupervised and unsupervised machine translation systems. |
| Outcome: | The proposed system achieves 22.5 BLEU points in English-to-German WMT 2014, 5.5 points more than the previous best unsupervised system, and 0.5 points more in the (supervised) shared task winner back in 2014. |
Efficient Neural Machine Translation for Low-Resource Languages via Exploiting Related Languages (2020.acl-srw)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) is a rapidly advancing MT paradigm that can be used to improve machine translation for many languages. |
| Approach: | They propose a technique called Unified Transliteration and Subword Segmentation to leverage language similarity while exploiting parallel data from related languages. |
| Outcome: | The proposed approach improves translation accuracy by 5 BLEU points over the standard Transformer-based NMT models. |
Mixed Multi-Head Self-Attention for Neural Machine Translation (D19-56)
Copied to clipboard
| Challenge: | Recent advances in neural machine translation have been made in the field of multi-head self-attention and there is no explicit mechanism to ensure that different attention heads capture different features. |
| Approach: | They propose a novel multi-head self-attention model which models not only global and local attention but also forward and backward attention in different attention heads. |
| Outcome: | The proposed model improves on WAT17 English-Japanese and IWSLT14 German-English translation tasks without increasing the number of parameters. |
Reinforcement Learning for Edit-Based Non-Autoregressive Neural Machine Translation (2024.naacl-srw)
Copied to clipboard
| Challenge: | Non-autoregressive (NAR) language models have a performance gap due to the large decoding space and difficulty in capturing dependency between target words accurately. |
| Approach: | They propose to use reinforcement learning to enhance the performance of edit-based NAR models by using stepwise reward maximization and episodic reward maximisation. |
| Outcome: | The proposed model outperforms autoregressive models in the evaluation of an edit-based model. |
A Copy Mechanism for Handling Knowledge Base Elements in SPARQL Neural Machine Translation (2022.findings-aacl)
Copied to clipboard
| Challenge: | Current architectures are unable to integrate knowledge base schema and handle questions unseen during training rendering them unusable outside the scope of topics covered in the training set. |
| Approach: | They propose to integrate a copy mechanism for neural SPARQL query generation by adding a knowledge base layer and a dynamic knowledge base vocabulary to two Seq2Seq architectures. |
| Outcome: | The proposed model outperforms existing models on state-of-the-art datasets and shows a significant increase in performance. |
YANMTT: Yet Another Neural Machine Translation Toolkit (2023.acl-demo)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is an end-to-end approach that provides stateof-the-art results for a variety of language pairs. |
| Approach: | They propose to build an open-source neural machine translation toolkit on top of HuggingFace's Transformers library and use it for pre-training and fine-tuning sequence-to-sequence models. |
| Outcome: | The proposed toolkit is built on top of the HuggingFace Transformers library and provides advanced features such as document/multi-source NMT, simultaneous NMT and mixtures-of-experts. |
Interrogating the Explanatory Power of Attention in Neural Machine Translation (D19-56)
Copied to clipboard
| Challenge: | Attention models are often used to justify the model’s decision in generating a token but it has not been rigorously established to what extent attention is a reliable source of information in NMT. |
| Approach: | They propose to use attention models to modify crucial aspects of the trained attention model to produce function and content words in the translation process. |
| Outcome: | The proposed models preserve function and content words in the translation process compared to state-of-the-art models. |
Moon IME: Neural-based Chinese Pinyin Aided Input Method with Customizable Association (P18-4)
Copied to clipboard
| Challenge: | a pinyin input method engine (IME) allows users to input Chinese into a computer by typing pinyan through the common keyboard. |
| Approach: | They present a pinyin IME that integrates neural machine translation and IR to offer amusive and customizable association ability. |
| Outcome: | The Moon IME integrates neural machine translation and IR to offer amusive association ability. |
UCSYNLP-Lab Machine Translation Systems for WAT 2019 (D19-52)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) has achieved stateof-the-art performance on various language pairs. |
| Approach: | They describe the UCSYNLP-Lab submission to WAT 2019 for Myanmar-English translation tasks in both directions. |
| Outcome: | The proposed translation system improves the performance of Myanmar-English translation tasks. |
Why Find the Right One? (2021.eacl-srw)
Copied to clipboard
| Challenge: | a new study investigates the impact of anaphoric one words in English on the neural machine translation process. |
| Approach: | They investigate the impact of anaphoric one words in English on the Neural Machine Translation process using English-Hindi as source and target language pair. |
| Outcome: | The proposed system performs poorly on sentences containing anaphoric ones compared to sentences involving regular, non-anaphorical ones . the results show that amongst the anamorphic words, the noun class is clearly much harder for NMT than the determinatives . |
Reproducible and Efficient Benchmarks for Hyperparameter Optimization of Neural Machine Translation Systems (2020.tacl-1)
Copied to clipboard
| Challenge: | Optimal versus suboptimal hyperparameters can lead to dramatic swings in system performance. |
| Approach: | They propose to use a library of pre-trained models for fast, low cost HPO experimentation and to propose metrics for evaluating HPO methods on NMT. |
| Outcome: | The proposed method uses a library of pre-trained models for fast, low cost experimentation on neural machine translation (NMT) . |
Learning to Generate Word- and Phrase-Embeddings for Efficient Phrase-Based Neural Machine Translation (D19-56)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) often fails in one-to-many translation, e.g., in the translation of multi-word expressions, compounds, and collocations. |
| Approach: | They propose a phrase-based NMT model that generates embeddings of words or phrases. |
| Outcome: | The proposed model performs on par with state-of-the-art phrase-based NMT. |
Songs Across Borders: Singable and Controllable Neural Lyric Translation (2023.acl-long)
Copied to clipboard
| Challenge: | Obtaining singable lyric translations can facilitate the globalization of the music publishing industry . |
| Approach: | They formalize lyric translation into a constrained translation problem and instantiate them to an English-Chinese system. |
| Outcome: | The proposed model achieves 99.85%, 99.00%, and 95.52% on length accuracy, rhyme accuracy, and word boundary recall. |
Sentiment Aware Neural Machine Translation (D19-52)
Copied to clipboard
| Challenge: | Sentiment ambiguous lexicons are used when context is absent in translations . most systems aim to produce one correct translation for a given source sentence . |
| Approach: | They propose a neural machine translation method that preserves sentiment in two sentiment scenarios and a method that embeds sentiment into a sentence. |
| Outcome: | The proposed method outperforms a baseline with sentiment-aware translations in both the BLEU score and translation accuracy. |
Granularity is crucial when applying differential privacy to text: An investigation for neural machine translation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Differential privacy (DP) is a method that is increasingly popular in NLP but the choice of granularity at which it is applied is often neglected. |
| Approach: | They propose to apply differential privacy (DP) to a neural machine translation dataset to protect individual data points. |
| Outcome: | The proposed method is more resistant to membership inference attacks than the document-level NMT system. |
Neural Machine Translation for Agglutinative Languages via Data Rejuvenation (2025.acl-srw)
Copied to clipboard
| Challenge: | Recent years, advances in Neural Machine Translation (NMT) heavily rely on large-scale parallel corpora. |
| Approach: | They propose to combine fine-grained inactive sample identification with target-side rejuvenation to improve translation quality from agglutinative languages. |
| Outcome: | The proposed framework improves on four low-resource agglutinative language tasks. |
Evaluating Explanation Methods for Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) has seen great success during recent years. |
| Approach: | They propose a metric that measures the fidelity of explanation methods on translation tasks . they use an efficient approximation to evaluate several explanation methods . |
| Outcome: | The proposed metric is efficient and can be used on translation tasks. |
On the differences between BERT and MT encoder spaces and how to address them in translation tasks (2021.acl-srw)
Copied to clipboard
| Challenge: | Various studies show that pretrained language models cannot replace encoders in neural machine translation despite their success in other tasks. |
| Approach: | They propose a supervised transformation from one into the other to improve the applicability of BERT in neural machine translation. |
| Outcome: | The proposed transformations show that they cannot replace encoders in MT despite their success in other tasks. |
Addressing Troublesome Words in Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) has weaknesses in handling lowfrequency and ambiguous words, which we refer to as troublesome words. |
| Approach: | They propose to use contextual memory to memorize which target words should be produced in which situations to translate troublesome words. |
| Outcome: | The proposed method outperforms baseline models on Chinese-to-English and English-to German translation tasks. |
MobileNMT: Enabling Translation in 15MB and 30ms (2023.acl-industry)
Copied to clipboard
| Challenge: | Existing work on NMT models is limited in storage, memory, computation and power consumption. |
| Approach: | They propose a mobile machine translation system that can translate in 15MB and 30ms on devices. |
| Outcome: | The proposed system can translate in 15MB and 30ms on mobile devices. |
Learning Source Phrase Representations for Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to machine translation have been shown to be effective for long sentences . however, the attentional network can't capture long-distance dependencies . |
| Approach: | They propose a multi-head attention mechanism which generates phrase representations from token representations and incorporates them into the Transformer translation model to enhance its ability to capture long-distance relationships. |
| Outcome: | The proposed model can be computed in parallel and improves on the WMT 14 tasks. |
Pre-training via Leveraging Assisting Languages for Neural Machine Translation (2020.acl-srw)
Copied to clipboard
| Challenge: | Sequence-to-sequence (S2S) pre-training with large monolingual data is not always available for the languages of interest (LOI). |
| Approach: | They propose to use monolingual corpora of other languages to complement the scarce monolingual LOI by script mapping (Chinese to Japanese) . Using only Chinese and French monolinguals, they improve Japanese-English translation quality by up to 8.5 BLEU in low-resource scenarios. |
| Outcome: | The proposed approach improves Japanese-English translation quality by up to 8.5 BLEU in low-resource scenarios. |
Checkpoint Reranking: An Approach to Select Better Hypothesis for Neural Machine Translation Systems (2020.acl-srw)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) has produced excellent results in the field of machine translation due to generation of high-quality translations for different language pairs. |
| Approach: | They propose a method of re-ranking the outputs of Neural Machine Translation systems by focusing on the decoder's ability to generate distinct tokens and without the use of any language model or data. |
| Outcome: | The proposed method achieves translation improvement up to +0.16 BLEU points over baseline. |
Attention Weights in Transformer NMT Fail Aligning Words Between Sequences but Largely Explain Model Predictions (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Using attention weights, we show that NMT models make alignment errors by relying on uninformative tokens from the source sequence. |
| Approach: | They propose to use attention weights to regulate alignment errors in NMT models . they propose methods that largely reduce the word alignment error rate compared to standard induced alignments from attention weighted tokens. |
| Outcome: | The proposed methods reduce the word alignment error rate compared to standard induced alignments from attention weights. |
What Works and Doesn’t Work, A Deep Decoder for Neural Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Deep learning has demonstrated performance advantages in a wide range of natural language processing tasks. |
| Approach: | They propose to deepen the decoder layer in a Transformer model to reduce the difficulty of deep learning. |
| Outcome: | The proposed method can deepen the model on both the encoder and decoder at the same time, resulting in a deeper model and improved performance. |
Multiscale Collaborative Deep Models for Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Neural machine translation models with deeper neural networks are difficult to train. |
| Approach: | They propose a MultiScale Collaborative framework to boost gradient back-propagation . they let each encoder block learn a fine-grained representation and enhance it . |
| Outcome: | The proposed framework outperforms baseline models on translation tasks with three translation directions and achieves a BLEU score of 30.56 on the English-to-German task. |
Norm-Based Curriculum Learning for Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Experimental results show that the proposed method outperforms strong baselines in terms of BLEU score (+1.17/+1.56) and training speedup (2.22x/3.33x). |
| Approach: | They propose a norm-based curriculum learning method that measures difficulty, competence and weight of a sentence in a word embedding. |
| Outcome: | The proposed method outperforms baselines in terms of BLEU score (+1.17/+1.56) and training speedup (2.22x/3.33x). |
Multi-Domain Neural Machine Translation with Word-Level Domain Context Discrimination (D18-1)
Copied to clipboard
| Challenge: | Experimental results on Chinese-English and English-French multi-domain translation tasks demonstrate the effectiveness of the proposed model. |
| Approach: | They propose to use mixed-domain parallel sentences to construct a unified model that allows translation to switch between different domains. |
| Outcome: | The proposed model distinguishes and exploits word-level domain contexts on Chinese-English and English-French translation tasks. |
Data Weighted Training Strategies for Grammatical Error Correction (2020.tacl-1)
Copied to clipboard
| Challenge: | Recent advances in the task of Grammatical Error Correction (GEC) have been driven by addressing data sparsity, both through new methods for generating large and noisy pretraining data and through the publication of small and higher-quality finetuning data in the BEA-2019 shared task. |
| Approach: | They propose to incorporate delta-log-perplexity, a type of example scoring, into a training schedule for Grammatical Error Correction (GEC) they perform experiments that shed light on the function and applicability of delta- log-perplicity. |
| Outcome: | The proposed methods incorporate delta-log-perplexity, a type of example scoring, into a training schedule for the task. |
Learning to Translate Ambiguous Terminology by Preference Optimization on Post-Edits (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Ambiguous terminology can make translation difficult, especially in corporate contexts. |
| Approach: | They propose to learn how to disambiguate terminology based on human post-edits . they use preference optimization to optimize for correctness using the term post-Edit . |
| Outcome: | The proposed framework improves term accuracy over a translation oriented LLM without significant losses in COMET score. |
Improving Neural Machine Translation Robustness via Data Augmentation: Beyond Back-Translation (D19-55)
Copied to clipboard
| Challenge: | Neural Machine Translation models are sensitive to noise in the input data. |
| Approach: | They propose new methods to extend limited noisy data and further improve NMT robustness to noise while keeping the models small. |
| Outcome: | The proposed methods extend limited noisy data and improve robustness to noise while keeping the models small. |
Bi-Directional Differentiable Input Reconstruction for Low-Resource Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | Existing work has addressed this problem by leveraging monolingual or multilingual data. |
| Approach: | They propose to introduce a differentiable reconstruction loss for neural machine translation to exploit the limited amounts of parallel text available in low-resource settings. |
| Outcome: | The proposed approach achieves small but consistent BLEU improvements on four language pairs in both translation directions and outperforms an alternative differentiable reconstruction strategy based on hidden states. |
ChrEn: Cherokee-English Machine Translation for Endangered Language Revitalization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Cherokee is a highly endangered Native American language spoken by the Cherokee people . there are only 2,000 fluent first language Cherokee speakers remaining in the world . |
| Approach: | They propose a Cherokee-English parallel dataset to facilitate machine translation between Cherokee and English. |
| Outcome: | The proposed dataset compares Cherokee-English and English-Cherokee machine translation systems . the results show that the datasets are low-resource and low-cost compared to other datasets . |
Near Human-Level Performance in Grammatical Error Correction with Hybrid Machine Translation (N18-2)
Copied to clipboard
| Challenge: | Currently, most effective GEC systems are based on phrase-based statistical machine translation. |
| Approach: | They combine two of the most popular approaches to automated Grammatical Error Correction (GEC) they create a hybrid GEC system that preserves the accuracy of SMT output and generates more fluent sentences . |
| Outcome: | The proposed system achieves state-of-the-art on the CoNLL-2014 and JFLEG benchmarks. |
Synthesizing Parallel Data of User-Generated Texts with Zero-Shot Neural Machine Translation (2020.tacl-1)
Copied to clipboard
| Challenge: | Neural machine translation systems are usually trained on clean parallel data, but the quality of translations is poor when translating noisy texts. |
| Approach: | They synthesize parallel data of UGT and exploit monolingual data to generate translations . they propose to use monolingual parallel data to train or adapt NMT systems . |
| Outcome: | The proposed approach improves the translation quality of noisy texts while making them more robust. |
Adaptive Nearest Neighbor Machine Translation (2021.acl-short)
Copied to clipboard
| Challenge: | kNN-MT uses pre-trained NMT model with token-level k-nearest-neighbor retrieval to improve translation accuracy. |
| Approach: | They propose a method that combines a pre-trained NMT model with token-level k-nearest-neighbor retrieval to improve translation accuracy. |
| Outcome: | The proposed method outperforms the existing model on four benchmark datasets and is open-source. |
Fast Nearest Neighbor Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Fast kNN-MT uses the entire corpus as the datastore for the nearest neighbor search . knn-MT is two-orders slower than vanilla MT models . |
| Approach: | They propose a fast kNN-MT model that uses the entire corpus as the datastore for nearest neighbor search. |
| Outcome: | The proposed model is two-orders faster than kNN-MT and is only two times slower than the standard model. |
Adaptive Multi-pass Decoder for Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | End-to-end neural machine translation (NMT) has attracted increasing attention in recent years. |
| Approach: | They propose an adaptive multi-pass decoder which introduces a flexible multi- pass polishing mechanism to extend the capacity of NMT via reinforcement learning. |
| Outcome: | The proposed architecture improves Chinese-English translation with 1.55 BLEU . the proposed architecture adopts a flexible multi-pass polishing mechanism . |
Dynamic Sentence Sampling for Efficient Training of Neural Machine Translation (P18-2)
Copied to clipboard
| Challenge: | Existing methods to train neural machine translation (NMT) use a fixed training procedure where each sentence is sampled once during each epoch. |
| Approach: | They propose to dynamically sample sentences to accelerate NMT training . a weight is assigned to each sentence based on the measured difference between training costs of two iterations. |
| Outcome: | Empirical results show that the proposed method can significantly accelerate training and improve NMT performance. |
Compositional Representation of Morphologically-Rich Input for Neural Machine Translation (P18-2)
Copied to clipboard
| Challenge: | Neural machine translation models are typically trained with fixed-size input and output vocabularies, which creates a bottleneck on their accuracy and generalization capability. |
| Approach: | They propose to replace the source-language embedding layer of NMT with a bi-directional recurrent neural network that generates compositional representations of the input at any desired level of granularity. |
| Outcome: | The proposed approach outperforms existing methods in a low-resource setting with five languages . the proposed approach consistently outperformed existing methods with a single word representation . |
Modeling Coherence for Neural Machine Translation with Dynamic and Topic Caches (C18-1)
Copied to clipboard
| Challenge: | Current neural machine translation systems translate a text sentence-by-sentence, ignoring cross-sentent links and dependencies. |
| Approach: | They propose a cache-based approach to modeling coherence for neural machine translation . they capture contextual information either from recently translated sentences or the entire document . |
| Outcome: | The proposed model improves on state-of-the-art translation models on many languages . it captures contextual information from recently translated sentences or the entire document . |
Improving Neural Machine Translation by Incorporating Hierarchical Subword Features (C18-1)
Copied to clipboard
| Challenge: | Using subwords, we find that the appropriate subword units for the three layers differ depending on the model . incorporating hierarchical subword features improves BLEU scores on the IWSLT evaluation datasets. |
| Approach: | They propose a method that expresses a word by combining "subwords" they propose to incorporate hierarchical subword features into a single embedding layer . |
| Outcome: | The proposed method improves BLEU scores on the IWSLT evaluation datasets. |
Quality Does Matter: A Detailed Look at the Quality and Utility of Web-Mined Parallel Corpora (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing web-mined corpora for low-resource languages have serious quality issues, especially for lowresource language pairs. |
| Approach: | They ranked each corpus according to a similarity measure and evaluated different portions of this ranked corpus. |
| Outcome: | The results show that the quality of web-mined corpora for low-resource languages is significantly different from human-curated corporats. |
Learning from Chunk-based Feedback in Neural Machine Translation (P18-2)
Copied to clipboard
| Challenge: | a common problem with explicit ratings of translations is that users are not qualified enough to provide reliable feedback for the whole sentence. |
| Approach: | They propose a way to learn from partial feedback in neural machine translation . they ask users to highlight a correct chunk of a translation based on partial feedback . |
| Outcome: | The proposed method outperforms sentence-based feedback by 2.61% BLEU absolute. |
A Comparison of Transformer and Recurrent Neural Networks on Multilingual Neural Machine Translation (C18-1)
Copied to clipboard
| Challenge: | Recent studies have shown that multilingual NMT models can handle more than one translation direction with a single system. |
| Approach: | They propose a multilingual neural machine translation model that can handle more than one translation direction with a single system. |
| Outcome: | The proposed model performs well in low-resource settings against bilingual systems. |
Machine Translation for Livonian: Catering to 20 Speakers (2022.acl-short)
Copied to clipboard
| Challenge: | Livonian is one of the most endangered languages in Europe with just a tiny handful of speakers and virtually no publicly available corpora. |
| Approach: | They aim to develop machine translation between Livonian and English using a linguistic similarity test and a dataset of parallel and monolingual data. |
| Outcome: | The proposed systems and the collected data, including a manually translated and verified translation benchmark, are publicly released via OPUS and Huggingface repositories. |
Using Semantic Similarity as Reward for Reinforcement Learning in Sentence Generation (P19-2)
Copied to clipboard
| Challenge: | Existing models for sentence generation use cross-entropy loss as the loss function . however, cross-etropy is unable to evaluate sentences as a whole and lacks flexibility . et al., 2018: a novel approach to improve sentence generation models . |
| Approach: | They propose a method to train a model using estimated semantic similarity between output and reference sentences to alleviate cross-entropy loss problems. |
| Outcome: | The proposed model improves the BLEU scores from the baseline LSTM NMT model. |
An Empirical Study on Adversarial Attack on NMT: Languages and Positions Matter (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to generating NMT adversarial examples inject perturbations into source sentences or target translations to improve the robustness of NMT models. |
| Approach: | They investigate adversarial attack on NMT from two aspects: languages (the source vs. the target language) and positions (front v. rear). |
| Outcome: | The proposed approach is more effective than adversarial attacks by sampling positions randomly or according to gradients. |
Improving Back-Translation with Uncertainty-based Confidence Estimation (D19-1)
Copied to clipboard
| Challenge: | Despite the success of low-resource neural machine translation, there is a data scarcity problem in many languages . large-scale, high-quality, and widecoverage bilingual corpora do not exist for most language pairs . |
| Approach: | They propose to quantify confidence of NMT models based on model uncertainty . they propose to use uncertainty-based confidence measures to improve back-translation . |
| Outcome: | The proposed model outperforms conventional statistical machine translation (SMT) on Chinese-English and English-German translation tasks. |
Why Skip If You Can Combine: A Simple Knowledge Distillation Technique for Intermediate Layers (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge distillation techniques are not suitable for deep learning tasks due to memory constraints. |
| Approach: | They propose to combine knowledge from a large teacher network into a student network (S) they propose to use a combinatorial mechanism to inject layer-level supervision from T to S . |
| Outcome: | The proposed model outperforms existing models in PortugueseEnglish, TurkishEnglish and EnglishGerman directions and students trained using it have 50% fewer parameters and can deliver comparable results to 12-layer teachers. |
Towards Linear Time Neural Machine Translation with Capsule Networks (D19-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) is an endto-end learning approach to machine translation. |
| Approach: | They propose a capsule network with dynamic routing for linear time Neural Machine Translation . they map the source sentence into a matrix with pre-determined size and apply a deep LSTM network to decode the target sequence from the source representation. |
| Outcome: | The proposed network achieves comparable results with the Transformer system on English-German and English-French tasks. |
Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translation (2023.eacl-main)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is becoming more accurate, but hallucinations are extremely pathological . previous work focused on artificial settings where the problem is amplified, disregarding some common types of hallucines . |
| Approach: | They propose a method for alleviating hallucinations at test time that significantly reduces the hallucinic rate. |
| Outcome: | The proposed method significantly reduces the hallucinatory rate in a natural setting. |
Iterative Dual Domain Adaptation for Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Empirical results on Chinese-English and English-German translation tasks demonstrate the effectiveness of our proposed framework. |
| Approach: | They propose an iterative dual domain adaptation framework for neural machine translation that uses multiple corpora to perform bidirectional translation knowledge transfer. |
| Outcome: | Empirical results on Chinese-English and English-German translation tasks demonstrate the effectiveness of the proposed framework. |
Multi-agent Learning for Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Experimental results show that training with more than one agent improves translation quality and improves accuracy. |
| Approach: | They propose to introduce diverse agents in an in- teractive updating process to train NMT models with an additional agent. |
| Outcome: | The proposed approach improves on NIST Chinese-English, IWSLT 2014 German- English, WMT 2014 English-German translation tasks and shows competitive performance on all tasks. |
Pivot-based Transfer Learning for Neural Machine Translation between Non-English Languages (D19-1)
Copied to clipboard
| Challenge: | Using parallel corpora, we train a single, direct NMT model for non-English language pairs. |
| Approach: | They propose three ways to increase the relation among source, pivot, and target languages in pre-training . they use additional adapter component to smoothly connect pre-trained encoder and decoder . |
| Outcome: | The proposed methods outperform multilingual models up to +2.6% BLEU in WMT 2019 French-German and German-Czech tasks. |
Context-Aware Monolingual Repair for Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | et al., 2018) show that human raters prefer corrected translations over the baseline ones. |
| Approach: | They propose a monolingual model to correct inconsistencies between sentences . they use monolingual document-level data to train the model . |
| Outcome: | The proposed model improves translations of contextual phenomena in English-Russian translation task. |
Long-Short Term Masking Transformer: A Simple but Effective Baseline for Document-level Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in deep learning have led to significant improvement of document-level neural machine translation (NMT). |
| Approach: | They propose a long-short term masking self-attention on top of the standard transformer to capture the long-range dependence and reduce the propagation of errors. |
| Outcome: | The proposed model captures the long-range dependence and reduces errors on two publicly available document-level datasets. |
Generating Diverse Translation from Model Distribution with Dropout (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing neural machine translation models lack diversity in their generation. |
| Approach: | They propose to generate diverse translations by deriving Bayesian models and sampling models from them for inference. |
| Outcome: | The proposed method makes a better trade-off between diversity and accuracy. |
On the Evaluation of Semantic Phenomena in Neural Machine Translation Using Natural Language Inference (N18-2)
Copied to clipboard
| Challenge: | Existing methods to investigate whether sentence representations from NMT systems capture distinct semantic phenomena are limited. |
| Approach: | They propose a process to investigate the extent to which sentence representations arising from neural machine translation systems encode distinct semantic phenomena. |
| Outcome: | The proposed model is suited to supporting inferences at the syntax-semantics interface, compared to anaphora resolution requiring world knowledge. |
Dynamic Past and Future for Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Neural machine translation models can benefit from modeling translated and untranslated source contents as recurrent states, but this less interpretable recurrence hinders their power to model dynamic updating of and contents during decoding. |
| Approach: | They propose to model the dynamic updating of and contents during decoding by explicitly separating source words into groups of translated and untranslated contents through parts-to-wholes assignment. |
| Outcome: | The proposed method achieves significant improvements over both Rnmt and Transformer by producing more adequate translations. |
Analyzing the Source and Target Contributions to Predictions in Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Neural Machine Translation models are influenced by two types of context, source and target, but none explicitly evaluates relative contribution to generation decision. |
| Approach: | They propose to adopt a variant of Layerwise Relevance Propagation which evaluates relative contributions to the generation decision by a proportion of token influence. |
| Outcome: | The proposed model can evaluate the relative contribution of source and target to the generation decision by using a variant of Layerwise Relevance Propagation (LRP) |
Improving Lexically Constrained Neural Machine Translation with Source-Conditioned Masked Span Prediction (2021.acl-short)
Copied to clipboard
| Challenge: | Accurate terminology translation is crucial for ensuring the practicality and reliability of neural machine translation systems. |
| Approach: | They propose a method to preserve terminology in translations as lexical constraints with or without a term dictionary at test time. |
| Outcome: | The proposed setup achieves consistent improvements on terminology and sentence-level translation for three domain-specific corpora in two language pairs. |
The Multilingual Microblog Translation Corpus: Improving and Evaluating Translation of User-Generated Text (2022.lrec-1)
Copied to clipboard
| Challenge: | a corpus of over 200,000 microblog translations supports translation of thirteen languages into English . large collections of parallel text, or bitext, are increasingly available in many languages . |
| Approach: | They propose a corpus of over 200,000 microblog posts that supports translation of thirteen languages into English. |
| Outcome: | The proposed corpus contains over 200,000 translations of microblog posts in 13 languages . fine-tuning showed significant improvements in translation quality . |
SwitchOut: an Efficient Data Augmentation Algorithm for Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Existing methods for data augmentation for text-based tasks such as machine translation are limited due to noise and noise. |
| Approach: | They propose a data augmentation policy with desirable properties as an optimization problem and propose 'SwitchOut' switchout randomly replaces words in both the source and target sentences with other random words from their corresponding vocabularies. |
| Outcome: | The proposed method outperforms strong alternatives such as word dropout on three translation datasets. |
A Testset for Context-Aware LLM Translation in Korean-to-English Discourse Level Translation (2025.coling-main)
Copied to clipboard
| Challenge: | Recent studies indicate that for high-resource languages, LLM surpasses encoder-decoder neural machine translation (NMT) models. |
| Approach: | They propose to construct a Korean-English discourse-level corpus with 600 text instances featuring six linguistic phenomena: lexical ambiguity, zero anaphora, slang, idiom, figurative language, and implicature. |
| Outcome: | The proposed corpus of 600 text instances features six linguistic phenomena, including lexical ambiguity, zero anaphora, slang, idiom, figurative language, and implicature. |
A Survey of Domain Adaptation for Neural Machine Translation (C18-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is a deep learning based approach for machine translation. |
| Approach: | They propose to use a deep learning approach to train machine translation in scenarios where large-scale parallel corpora are available. |
| Outcome: | The proposed approach yields the state-of-the-art translation performance in resource rich scenarios. |
Data Selection Curriculum for Neural Machine Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Neural Machine Translation models are typically trained on heterogeneous data that are concatenated and randomly shuffled. |
| Approach: | They propose a two-stage curriculum training framework where a NMT model is fine-tuned on subsets of data, selected by deterministic scoring and online scoring. |
| Outcome: | The proposed framework improves on six language pairs comprising low- and high-resource languages and shows up to +2.2 BLEU improvement and faster convergence. |
Multilingual Neural Machine Translation: Can Linguistic Hierarchies Help? (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Multilingual Neural Machine Translation (MNMT) trains a single model that supports translation between multiple languages . transferring knowledge from a diverse set of languages degrades the translation performance due to negative transfer. |
| Approach: | They propose a hierarchical knowledge distillation approach to train multilingual models . they use typological features and phylogeny to overcome negative transfer issue . |
| Outcome: | The proposed approach avoids negative transfer effect by capitalising on language groups generated according to typological features and phylogeny of languages. |
When a Good Translation is Wrong in Context: Context-Aware Machine Translation Improves on Deixis, Ellipsis, and Lexical Cohesion (P19-1)
Copied to clipboard
| Challenge: | et al., 2018: translation errors due to the lack of extra-sentential context are becoming more and more noticeable among otherwise adequate translations. |
| Approach: | They propose a context-aware translation model that uses sentence-level data to identify inconsistencies . standard metrics are not sensitive to improvements in consistency in document-level translations . |
| Outcome: | The proposed model shows major gains over baseline without sacrificing performance . standard metrics are not sensitive to improvements in document-level translations . |
Dense Information Flow for Neural Machine Translation (N18-1)
Copied to clipboard
| Challenge: | Recent advances in deep neural networks have improved learning performance for NMT . Residual connections allow features from previous layers to be accumulated to the next layer easily. |
| Approach: | They propose a densely connected NMT architecture that can train more efficiently for NMT. |
| Outcome: | The proposed architecture improves learning performance and attention quality on multiple datasets. |
Morphology Aware Source Term Masking for Terminology-Constrained NMT (2024.findings-eacl)
Copied to clipboard
| Challenge: | Recent research in terminology-constrained NMT systems focuses on data-driven approaches to generating translations. |
| Approach: | They propose a method that appends target term lemmas to their corresponding source terms in the input sentence while retaining essential grammatical information. |
| Outcome: | The proposed method improves on the “copy-and-inflect” method in two translation directions with different levels of source morphological complexity. |
Competence-based Curriculum Learning for Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | Existing NMT systems require specialized heuristics and large batch sizes. |
| Approach: | They propose a curriculum learning framework for NMT that reduces training time and costs . framework consists of a principled way of deciding which training samples are shown to the model . |
| Outcome: | The proposed framework can reduce training time and improve performance of recurrent neural network models and Transformers. |
Effective Cross-lingual Transfer of Neural Machine Translation Models without Shared Vocabularies (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to transfer a pretrained NMT model to a new, unrelated language without shared vocabularies are limited to cognate languages. |
| Approach: | They propose to transfer a pretrained NMT model to a new, unrelated language without shared vocabularies by using cross-lingual word embedding and injecting artificial noises. |
| Outcome: | The proposed methods outperform multilingual joint training by a large margin in five low-resource translation tasks. |
Modeling Concentrated Cross-Attention for Neural Machine Translation with Gaussian Mixture Model (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Dot-product attention only considers the pair-wise correlation between words, resulting in dispersion when dealing with long sentences and neglecting source neighboring relationships. |
| Approach: | They propose to model concentrated attention in cross-attention using a Gaussian Mixture Model to model cross- attention in a language model. |
| Outcome: | Experiments on three datasets show that the proposed method outperforms the baseline and has significant improvement on alignment quality, N-gram accuracy, and long sentence translation. |
Consistency by Agreement in Zero-Shot Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | In this paper, we focus on zero-shot generalization—a challenging setup that tests models on translation directions they have not been optimized for at training time. |
| Approach: | They propose a method that allows for a consistent agreement-based training method that encourages the model to produce equivalent translations of parallel sentences in auxiliary languages. |
| Outcome: | The proposed model improves on public zero-shot translation benchmarks without loss of performance on supervised translation directions. |
Handling Homographs in Neural Machine Translation (N18-1)
Copied to clipboard
| Challenge: | Existing methods for MT have problems with translating homographs, as it is difficult to select the correct translation based on the context. |
| Approach: | They propose to model the context of the input word with context-aware word embeddings that help to differentiate the word sense before feeding it into the encoder. |
| Outcome: | The proposed models improve translation accuracy and BLEU score on three language pairs. |
Quality Beyond A Glance: Revealing Large Quality Differences Between Web-Crawled Parallel Corpora (2025.coling-main)
Copied to clipboard
| Challenge: | Parallel corpora play a vital role in advanced multilingual natural language processing tasks, notably in machine translation (MT). |
| Approach: | They manually and automatically evaluated four well-known publicly available parallel corpora across eleven language pairs. |
| Outcome: | The results show that the four well-known parallel corpora have a substantial amount of noisy sentence pairs, while CCMatrix and CCAligned have low quality sentences. |
On the Word Alignment from Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Prior researches suggest that neural machine translation (NMT) captures word alignment through its attention mechanism, however, attention may fail to capture word alignment for some NMT models. |
| Approach: | They propose two methods to induce word alignment which are general and agnostic to specific NMT models. |
| Outcome: | The proposed methods induce much better word alignment than attention. |
Representation Collapse in Machine Translation Through the Lens of Angular Dispersion (2026.findings-eacl)
Copied to clipboard
| Challenge: | Effective training of Transformer models for sequential language tasks is difficult due to various forms of collapse of the internal representations learned. |
| Approach: | They propose to use angular dispersion to analyze representation collapse at different levels of discrete and continuous transformers throughout training. |
| Outcome: | The proposed method mitigates collapse and improves translation quality. |
Putting words into the system’s mouth: A targeted attack on neural machine translation using monolingual data poisoning (2021.findings-acl)
Copied to clipboard
Jun Wang, Chang Xu, Francisco Guzmán, Ahmed El-Kishky, Yuqing Tang, Benjamin Rubinstein, Trevor Cohn
| Challenge: | Neural machine translation systems are known to be vulnerable to adversarial test inputs, however, they are also vulnerable to training attacks. |
| Approach: | They propose a poisoning attack in which a malicious adversary inserts a small poisoned sample of monolingual text into a training set of a system trained using back-translation. |
| Outcome: | The proposed attack is based on two methods that can be used to craft poisoned examples. |
Cross-Corpora Evaluation and Analysis of Grammatical Error Correction Models — Is Single-Corpus Evaluation Enough? (N19-1)
Copied to clipboard
| Challenge: | Existing studies have evaluated grammatical error correction models on a single corpus, but the evaluation is incomplete because the task difficulty varies depending on the corpus and conditions such as proficiency levels of the writers and essay topics. |
| Approach: | They evaluate the performance of several GEC models against various learner corpora and compare their rankings against the corpus. |
| Outcome: | The evaluation of several models against learner corpora shows that the models’ rankings vary depending on the corpus, indicating that single-corpus evaluation is insufficient for GEC models. |
Simpler and Faster Learning of Adaptive Policies for Simultaneous Translation (D19-1)
Copied to clipboard
| Challenge: | Recent work on simultaneous translation is difficult because of its latency and quality. |
| Approach: | They propose a supervised-learning framework to learn adaptive policies from parallel text sequences . they use a model that predicts when a target word is read or WRITE if context provides enough information . |
| Outcome: | Experiments on German=>English show that the proposed method can learn flexible policies with better BLEU scores and similar latencies compared to previous work. |
PROSE: A Pronoun Omission Solution for Chinese-English Spoken Language Translation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Pro-drop (‘pronoun-dropping’) language requires NMT systems to recover omitted pronouns, but this task lacks sufficient datasets for benchmarking . |
| Approach: | They propose a benchmarking method that leverages the semantic embedding of dropped pronouns to augment training pairs to alleviate the negative impact introduced by pro-drop . |
| Outcome: | The proposed method outperforms existing methods regarding omitted pronoun retrieval and overall translation quality on four Chinese-English translation corpora. |
Handling Syntactic Divergence in Low-resource Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to neural machine translation (NMT) are dependent on limited parallel data, and can be difficult to use for many language pairs. |
| Approach: | They propose a method where target-language sentences are re-ordered to match the order of the source and used as an additional source of training-time supervision. |
| Outcome: | The proposed method improves on simulated low-resource Japanese-to-English and real low-demand Uyghur-to English scenarios. |
Exploiting Pre-Ordering for Neural Machine Translation (L18-1)
Copied to clipboard
| Challenge: | Existing studies have shown that Neural Machine Translation suffers from the problems that some source words are mistakenly translated for multiple times . |
| Approach: | They propose a pre-ordering approach to solve the under-translation problem by pre-ordnanced source sentences and position embedding to enhance monotone translation. |
| Outcome: | The proposed method significantly improves translation quality by 2.43 BLEU points on Chinese-to-English translation. |
Dynamic Oracle for Neural Machine Translation in Decoding Phase (L18-1)
Copied to clipboard
| Challenge: | Existing methods to improve NMT performance but there is a discrepancy between training and inference when decoding. |
| Approach: | They propose to use Scheduled Sampling to reduce the discrepancy between training and inference in NMT when decoding to mitigate the discrépancy. |
| Outcome: | The proposed methods improve translation quality over standard NMT system. |
One Sentence One Model for Neural Machine Translation (L18-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is a new state of the art that can produce better results than traditional statistical machine translation. |
| Approach: | They propose a dynamic neural network which learns a general network as usual and fine-tunes it for each test sentence. |
| Outcome: | The proposed method improves translation performance when similar sentences are available. |
Examining the Tip of the Iceberg: A Data Set for Idiom Translation (L18-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) has been widely used in recent years with significant improvements for many language pairs. |
| Approach: | They propose to use a large-scale data set to evaluate idiom translation in GermanEnglish. |
| Outcome: | The proposed dataset is used to perform preliminary NMT experiments on idiom translation in GermanEnglish. |
Unveiling the Power of Source: Source-based Minimum Bayes Risk Decoding for Neural Machine Translation (2025.acl-long)
Copied to clipboard
| Challenge: | Maximum a posteriori decoding aims to maximize the estimated posterior probability, but high estimated probability does not always lead to high translation quality. |
| Approach: | They propose a method that seeks hypotheses with the highest expected utility by using quasi-sources as “support hypothese . they propose sMBR decoding which utilizes a reference-free quality estimation metric as the utility function. |
| Outcome: | The proposed approach outperforms QE reranking and the standard MBR decoding. |
Encoders Help You Disambiguate Word Senses in Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Neural machine translation models can perform word sense disambiguation (WSD) however, it is unclear which component dominates the process of disambiguating words. |
| Approach: | They evaluate hidden states and investigate distributions of self-attention in NMT encoders and decoders to disambiguate word senses. |
| Outcome: | The proposed model outperforms encoder hidden states on large datasets . the model outpersforms decoders on large data sets . |
Source and Target Bidirectional Knowledge Distillation for End-to-end Speech Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | End-to-end speech translation models can be trained to leverage source text . however, since the input modalities are different, it is difficult to leverage the source text successfully. |
| Approach: | They propose to leverage source transcriptions via pre-training and joint training with ASR and NMT tasks. |
| Outcome: | The proposed model predicts paraphrased transcriptions as an auxiliary task with a single decoder. |
A Comparison between Pre-training and Large-scale Back-translation for Neural Machine Translation (2021.findings-acl)
Copied to clipboard
| Challenge: | BERT is a promising technique to improve NMT, but how it outperforms standard NMT is understudied. |
| Approach: | We compare MT engines trained with pre-trained BERT and back-translation with incrementally larger amounts of data. |
| Outcome: | The proposed technique outperforms standard NMT models on morphology and syntax. |
Distill The Image to Nowhere: Inversion Knowledge Distillation for Multimodal Machine Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on multimodal machine translation (MMT) have focused on the fusion and alignment of images and texts to improve MMT. |
| Approach: | They propose an image-free inference framework that supports image-based inference via an inversion knowledge distillation scheme. |
| Outcome: | The proposed framework is the first to rival or surpass image-must frameworks on the multimodal translation benchmark. |
One Size Does Not Fit All: Comparing NMT Representations of Different Granularities (N19-1)
Copied to clipboard
| Challenge: | Recent work has shown that contextualized word representations are a viable alternative to simple word prediction tasks. |
| Approach: | They propose to use subword units and characters to model morphology, syntax, and semantics instead of word embeddings. |
| Outcome: | The proposed representations are better for modeling syntax and more robust to noisy input. |
A Lightweight Mixture-of-Experts Neural Machine Translation Model with Stage-wise Training Strategy (2024.findings-naacl)
Copied to clipboard
| Challenge: | Using mixture-of-experts (MoE) to deal with language heterogeneity is a challenge in neural machine translation (NMT). |
| Approach: | They propose a lightweight MoE-based NMT model that is trained via an elaborate stage-wise training strategy. |
| Outcome: | The proposed model achieves stable improvements in translation tasks by introducing fewer extra parameters compared to baseline models. |
Bitext Mining Using Distilled Sentence Representations for Low-Resource Languages (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a new study aims to extend multilingual representation learning beyond the hundred most frequent languages . current work on multilingual sentence representations has focused on training one model which handles all languages of interest . |
| Approach: | They propose a teacher-student approach to extend existing monolingual sentence embedding space to new languages. |
| Outcome: | The proposed model outperforms the original LASER encoder in 44 African languages . the model can be used to train multiple languages and learn new languages if they have the same training data . |
Contrastive Decoding Reduces Hallucinations in Large Multilingual Machine Translation Models (2024.eacl-long)
Copied to clipboard
| Challenge: | Hallucinations occur when the target side sentence is detached from the source side sentence, or in other words, when there is a low contribution of the source sentence to the generation of the target sentence. |
| Approach: | They propose to use Contrastive Decoding to maximise the log-likelihood difference between a model and the same model with reduced contribution from the encoder outputs. |
| Outcome: | The proposed algorithm maximises the log-likelihood difference between a model and the same model with reduced contribution from the encoder outputs. |
Alignment verification to improve NMT translation towards highly inflectional languages with limited resources (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing approaches to improve translation quality using limited training data are phrase-based and syntax-based approaches. |
| Approach: | They propose to combine a neural MT system with an open source module to improve translation quality. |
| Outcome: | The proposed method improves translation quality over the best individual NMT and the standard ensemble system provided in the Marian-NMT system. |
Active Learning Approaches to Enhancing Neural Machine Translation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | a limited human translation budget is required to train neural machine translation models. |
| Approach: | They propose to integrate active learning into neural machine translation techniques . they propose a word frequency based acquisition function and an uncertainty based method . |
| Outcome: | The proposed method outperforms other acquisition functions on a limited human translation budget. |
Controlling Styles in Neural Machine Translation with Activation Prompt (2023.findings-acl)
Copied to clipboard
| Challenge: | Earlier studies on controlling styles in neural machine translation (NMT) have focused on regulating the level of formality, but they still encounter two major challenges. |
| Approach: | They propose a method to control the style of neural machine translation by retrieving prompts from stylized monolingual corpus. |
| Outcome: | The proposed method can control the style of translation and achieve remarkable performance. |
Reliability and Learnability of Human Bandit Feedback for Sequence-to-Sequence Reinforcement Learning (P18-1)
Copied to clipboard
| Challenge: | Recent work has shown that reinforcement learning (RL) can be scaled to games with large state-action spaces, achieving human-level performance or even superhuman performance. |
| Approach: | They propose to use bandit feedback to improve sequence-to-sequence learning by simulating reward signals by evaluation metrics such as BLEU, F1-score, or ROUGE. |
| Outcome: | The proposed methods improve performance even from small amounts of human feedback, pointing to a great potential for applications at larger scale. |
Simple, Scalable Adaptation for Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Recent advances in deep learning have led to significantly improved quality on Neural Machine Translation (NMT) however, performance on out-of-domain data or low resource languages remains poor. |
| Approach: | They propose a simple yet efficient approach for adapting pre-trained models to multiple tasks simultaneously. |
| Outcome: | The proposed approach is on par with full fine-tuning on domain adaptation and massively multilingual NMT on a massively multilingual dataset. |
How Much Attention Do You Need? A Granular Analysis of Neural Machine Translation Architectures (P18-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) has been replaced by convolutional or self-attentional approaches. |
| Approach: | They propose an architecture definition language that allows for a flexible combination of common building blocks. |
| Outcome: | The proposed architectures can bring recurrent and convolutional models close to the Transformer architecture, but not using self-attention. |
Investigating Multilingual NMT Representations at Scale (D19-1)
Copied to clipboard
| Challenge: | Multilingual Neural Machine Translation models have shown success in transfer learning settings, but their mode of transfer remains elusive. |
| Approach: | They propose to use a representation similarity framework to compare multilingual representations using a SVCCA representation similar to the previous work. |
| Outcome: | The proposed model can be used to compare representations across languages and layers. |
Learning Confidence for Transformer-based Neural Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | A well-calibrated confidence estimate is not sufficient for neural machine translation (NMT) where probabilities from softmax distribution fail to describe when the model is probably mistaken. |
| Approach: | They propose an unsupervised confidence estimate learning jointly with the training of a neural machine translation model to quantify confidence. |
| Outcome: | The proposed model outperforms standard label smoothing and can predict failures in two real-world scenarios. |
Hierarchical Modeling of Global Context for Document-Level Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Document-level machine translation (MT) remains challenging due to the difficulty in efficiently using document context. |
| Approach: | They propose a hierarchical model to learn document context for document-level neural machine translation . they use a sentence encoder to capture intra-sentence dependencies and a document encoder . |
| Outcome: | The proposed model significantly improves document-level translation performance over strong baselines. |
Data Rejuvenation: Exploiting Inactive Training Examples for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Large-scale training datasets make training neural machine translation models difficult. |
| Approach: | They propose to identify inactive training examples which contribute less to the model performance and introduce data rejuvenation to improve NMT models' training. |
| Outcome: | The proposed framework stabilizes and accelerates the training process of NMT models, resulting in models with better generalization capability. |
What Knowledge Is Needed? Towards Explainable Memory for kNN-MT Domain Adaptation (2023.findings-acl)
Copied to clipboard
| Challenge: | kNN-MT builds an external datastore, which saves all target language token occurrences in the parallel corpus. |
| Approach: | They propose a new paradigm for domain adaptation by building an external datastore which usually saves all target language token occurrences in the parallel corpus. |
| Outcome: | The proposed model can be easily pruned according to local correctness, and it is more explainable. |
Self-Supervised Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) methods relied on the availability of high-quality parallel corpora. |
| Approach: | They propose a method where an emergent NMT system is used for selecting training data and learning internal NMT representations. |
| Outcome: | The proposed method achieves BLEU scores of 29.21 (en2fr) and 27.36 (fr2en) on newstest2014 using English and French Wikipedia data for training. |
Analyzing Challenges in Neural Machine Translation for Software Localization (2023.eacl-main)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is a new form of machine translation that reduces the post-editing time of human annotators. |
| Approach: | They propose to use a novel multilingual UI corpus collection to test NMT for user interfaces. |
| Outcome: | The proposed test set evaluates state-of-the-art methods on a UI translation task from English to German and identifies its limitations. |
Measuring and Mitigating Name Biases in Neural Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Neural machine translation systems exhibit problematic biases, such as stereotypical gender bias in occupation terms. |
| Approach: | They propose a method to reduce biases in person name translations by randomly switching entities during translation. |
| Outcome: | The proposed method eliminates the problem without any effect on translation quality. |
Contrastive Token Learning with Similarity Decay for Repetition Suppression in Machine Translation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is pivotal for crosslingual conversation and trade . traditional solutions that penalize text redundancy or token reoccurrence have shown limited efficacy . |
| Approach: | They propose an algorithm that modulates suppression of tokens dynamically, informed by attention weights and inter-token distances. |
| Outcome: | The proposed algorithm outperforms existing methods in precision and generalizability. |
Machine Translationese: Effects of Algorithmic Bias on Linguistic Complexity in Machine Translation (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing studies have shown that existing models amplify biases observed in training data. |
| Approach: | They propose to use MT and NLP to amplify biases observed in training data to investigate how bias amplification might affect language in a broader sense. |
| Outcome: | The proposed model amplifys biases observed in training data and could lead to an artificially impoverished language, the authors show. |
Improving Neural Machine Translation with Neural Syntactic Distance (N19-1)
Copied to clipboard
| Challenge: | Neural syntactic distance (NSD) is used to represent constituent trees using a sequence whose length is identical to the number of words in the sentence. |
| Approach: | They propose five strategies to improve NMT with explicit use of syntactic information . et al., 2014) propose a set of five strategies that incorporate syntastic information into the encoder and/or decoder of the baseline model. |
| Outcome: | The proposed strategies improve translation performance of the baseline model (+2.1 (En–Ja), +1.3 (Ja–En), +1.2 (En-Ch), and +1.0 (Ch–En) BLEU. |
Confidence Based Bidirectional Global Context Aware Training Framework for Neural Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing studies focus on how to effectively exploit bidirectional global contexts in neural machine translation models. |
| Approach: | They propose a Confidence Based Bidirectional Global Context Aware training framework for NMT . they incorporate bidirectional global context to the NMT model on unconfidently-predicted target words . |
| Outcome: | The proposed framework improves the NMT model on three large-scale translation datasets by +1.02, +0.57 BLEU scores. |
A Semi-supervised Approach to Generate the Code-Mixed Text using Pre-trained Encoder and Transfer Learning (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to train neural network-based models for code-mixing are limited due to language specificity of code-mixed text. |
| Approach: | They propose a deep learning approach to generate code-mixed text from English to multiple languages without any parallel data. |
| Outcome: | The proposed approach generates a code-mixed text from English to multiple languages without any parallel data. |
CSP:Code-Switching Pre-training for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to train neural machine translation models are data-hungry and low-resource . et al., 2018; Radford e.t., 2019; Yang ee.,2019) proposes a new pre-training method for NMT . |
| Approach: | They propose a new pre-training method which randomly replaces some words in the input sentence with their translation words in target language. |
| Outcome: | The proposed method improves on unsupervised and supervised NMT models by making full use of monolingual corpora. |
Towards Enhancing Faithfulness for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) has achieved great success due to the ability to generate high-quality sentences. |
| Approach: | They propose a training strategy with a multi-task learning paradigm to build a faithfulness enhanced NMT model. |
| Outcome: | The proposed model can generate high-quality sentences that are very close to natural language. |
Context-aware Neural Machine Translation with Mini-batch Embedding (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing models that translate sentences in sentence-by-sentence manner do not consider the inter-sentent context in a language translation model. |
| Approach: | They propose to use mini-batch embedding to represent the features of sentences in a mini-Batch by choosing sentences from the same document and then embedd it into the model. |
| Outcome: | The proposed method outperforms baseline models and improves writing style or terminology to fit the document’s context. |
Uncertainty-Aware Semantic Augmentation for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for neural machine translation only observe one source sentence at training time . this discrepancy in data distribution leads to a formidable learning challenge . |
| Approach: | They propose an uncertainty-aware semantic augmentation approach to capture universal semantic information among multiple source sentences and enhance hidden representations with this information. |
| Outcome: | The proposed approach outperforms baseline and existing methods on translation tasks. |
Encoder and Decoder, Not One Less for Pre-trained Language Model Sponsored NMT (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for neural machine translation (NMT) use encoder-only enhancement or rely on specific multilingual PLMs. |
| Approach: | They propose a monolingual PLM-sponsored NMT model that lets both encoder and decoder enjoy PLM enhancement. |
| Outcome: | The proposed model improves on IWSLT14 En-De, De-En, WMT14 En -De and En-Fr tasks and achieves new state-of-the-art. |
Breaking the Corpus Bottleneck for Context-Aware Neural Machine Translation with Cross-Task Pre-training (2021.acl-long)
Copied to clipboard
| Challenge: | Context-aware neural machine translation (NMT) remains challenging due to the lack of large-scale document-level parallel corpora. |
| Approach: | They propose to use large-scale parallel datasets and source-side monolingual documents to improve context-aware neural machine translation. |
| Outcome: | The proposed model can be used to translate both sentences and documents on four translation tasks. |
Neural Machine Translation with Contrastive Translation Memories (2022.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that retrieval-augmented NMT model obtains substantial improvements over strong baselines in the benchmark dataset. |
| Approach: | They propose a retrieval-augmented NMT model that is holistically similar to the source sentence while individually contrastive to each other. |
| Outcome: | The proposed model improves on baselines in the translation task. |
Chinese-Portuguese Machine Translation: A Study on Building Parallel Corpora from Comparable Texts (L18-1)
Copied to clipboard
| Challenge: | Chinese and Portuguese are very populous languages, but there is not much parallel corpora in the Chinese-Portuguese language pair. |
| Approach: | They propose to curate Chinese-Portuguese parallel corpora and evaluate their quality . they extract bilingual data from government websites and use Phrased-Based Machine Translation (PBMT) and Neural Machine Translation models to build large corpus. |
| Outcome: | The proposed method can be used as a benchmark for future Chinese-Portuguese MT systems. |
Do Multilingual Neural Machine Translation Models Contain Language Pair Specific Attention Heads? (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent studies on multilingual representations focus on whether there is an emergence of language-independent representations or whether multilingual models partition their weights among different languages. |
| Approach: | They analyze encoder self-attention and encoder-decoder attention heads in a multilingual neural translation model. |
| Outcome: | The proposed model is based on a multilingual neural translation model with a language-independent representation. |
Recurrent Attention for Neural Machine Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent research questions the importance of dot-product self-attention in Transformer models and shows that most attention heads learn simple positional patterns. |
| Approach: | They propose a novel mechanism to replace dot-product self-attention with a recurrent atteNtion mechanism that directly learns attention weights without token-to-token interaction. |
| Outcome: | The proposed model outperforms the Transformer model on translation tasks with fewer parameters and inference time. |
A Self-Distillation Recipe for Neural Machine Translation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for Neural Machine Translation (NMT) have been proven effective in improving the performance of computer vision tasks without pre-training a teacher. |
| Approach: | They propose a rank-order augmented Pearson correlation loss and an iterative distillation method to prevent the discrepancy of predictions between the student and a stronger teacher from disturbing the training. |
| Outcome: | The proposed method can lead to significant improvements over the strong Transformer baseline on low/middle/high-resource tasks, obtaining comparable or better performance with fewer layers. |
Exploring Unsupervised Pretraining Objectives for Machine Translation (2021.findings-acl)
Copied to clipboard
| Challenge: | Unsupervised cross-lingual pretraining has significantly reduced the need for large parallel data. |
| Approach: | They compare unsupervised cross-lingual pretraining with masking and reconstructing inputs in the decoder to produce real sentences. |
| Outcome: | The proposed methods produce inputs resembling real (full) sentences, by reordering and replacing words based on their context. |
Encouraging Lexical Translation Consistency for Document-Level Neural Machine Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show document-level neural machine translation improves lexical consistency . inconsistent translations tend to confuse readers in some cases . |
| Approach: | They propose to use a word link to obtain a document word link and an auxiliary loss function to constrain that their translation should be consistent. |
| Outcome: | The proposed approach improves translation consistency on ChineseEnglish and EnglishFrench translation tasks. |
Unlocking Parameter-Efficient Fine-Tuning for Low-Resource Language Translation (2024.findings-naacl)
Copied to clipboard
| Challenge: | Parameter-efficient fine-tuning (PEFT) methods are important in low-resource language (LRL) Neural Machine Translation (NMT) but their practical effectiveness varies significantly across different languages. |
| Approach: | They evaluated the performance of 8 parameters-efficient fine-tuning methods with 15 architectures using the SacreBLEU score. |
| Outcome: | The Houlsby+Inversion adapter outperforms the baseline architectures in both in-domain and out-domain tests and the Houlson+Inverter achieves the best performance overall. |
Tailoring Neural Architectures for Translating from Morphologically Rich Languages (C18-1)
Copied to clipboard
| Challenge: | A morphologically complex word is a hierarchical constituent with meaning-preserving subunits, so word-based models which rely on surface forms might not be powerful enough to translate such structures. |
| Approach: | They propose a neural architecture which is designed to deal with morphological complexities on the source side and redesign the decoder accordingly to benefit from such information. |
| Outcome: | The proposed model outperforms existing subword- and character-based architectures and showed significant improvements on translating from German, Russian, and Turkish into English. |
Prevent the Language Model from being Overconfident in Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Neural Machine Translation models are based on partial translation and a language model that predicts the next token based only on partial. |
| Approach: | They propose a Margin-based Token-level Objective and a Sentence-level Goal to maximize the Margin . they propose to model the next token based on partial translation . |
| Outcome: | The proposed approach improves translation adequacy and fluency on English-to-German, Chinese-to English and French translation tasks. |
Neural Machine Translation Incorporating Named Entity (C18-1)
Copied to clipboard
| Challenge: | Conventional NMT models have difficulty translating words with multiple meanings because of the high ambiguity. |
| Approach: | They propose a neural machine translation model that incorporates named entity (NE) tags of source-language sentences to reduce the difficulty in translating multiple meanings. |
| Outcome: | The proposed model achieves 3.11 point improvement in bilingual evaluation understudy (BLEU) on English-to-Japanese translation task with the ASPEC, and English- to-Bulgarian and English to-Romanian translation tasks with the Europarl corpus. |
Adversarial Grammatical Error Correction (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results show that adversarial-GEC can achieve competitive GEC quality compared to NMT-based baselines. |
| Approach: | They propose an adversarial approach to Grammatical Error Correction using a transformer-based model and a sentence-pair classification model. |
| Outcome: | The proposed approach achieves competitive GEC quality compared to baselines. |
Deconvolution-Based Global Decoding for Neural Machine Translation (C18-1)
Copied to clipboard
| Challenge: | Existing models for Neural Machine Translation (NMT) use Recurrent Neural Network (RNN) to generate translation word by word following a sequential order. |
| Approach: | They propose a Neural Machine Translation (NMT) model that decodes the sequence with the guidance of its structural prediction of the target-side context. |
| Outcome: | The proposed model is more competitive compared with the state-of-the-art methods and reduces repetition with the instruction from the target-side context for decoding. |
Fix-Filter-Fix: Intuitively Connect Any Models for Effective Bug Fixing (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches for bug fixing lack generality and use only textual or structured information. |
| Approach: | They propose an intuitive yet effective general framework called Fix-Filter-Fix for bug fixing that connects models with their filter mechanism to filter out the last model’s unchanged fix to the next. |
| Outcome: | The proposed framework can quantify and accurately calculate the lifting effect of the model. |
Reference Network for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) generates translations in isolation, resulting in translation inconsistency and ambiguity. |
| Approach: | They propose to incorporate referring process into translation decoding of NMT by using local coordinates coding to obtain global context vectors containing monolingual and bilingual contextual information. |
| Outcome: | The proposed model improves translation quality with lightweight computation cost on Chinese-English and English-German translation tasks. |
Look Harder: A Neural Machine Translation Model with Hard Attention (P19-1)
Copied to clipboard
| Challenge: | Soft-attention based Neural Machine Translation models attend all the words in the source sequence for each target token, which makes them ineffective for long sequence translation. |
| Approach: | They propose a hard-attention based NMT model which selects a subset of source tokens for each target token to effectively handle long sequence translation. |
| Outcome: | The proposed model performs better on long sequences and achieves significant improvement on English-German and English-French translation tasks compared to soft-attention based models. |
Robust Neural Machine Translation with Joint Textual and Phonetic Embedding (P19-1)
Copied to clipboard
| Challenge: | Neural machine translation models are sensitive to noises in input sentences . one special kind of noise is the homophone noise, where words are replaced by other words with similar pronunciations. |
| Approach: | They propose to embed phonetic and textual information into neural machine translation datasets to improve robustness to homophone noises. |
| Outcome: | The proposed method improves the robustness of neural machine translation to homophone noises on clean test sets. |
Sentence-Level Agreement for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Empirical results show that a sentence-level agreement module can significantly improve the performance of neural machine translation (NMT) |
| Approach: | They propose a sentence-level agreement module to minimize the difference between the representation of source and target sentences. |
| Outcome: | Empirical results show the proposed agreement module significantly improves translation performance. |
Multilingual Unsupervised NMT using Shared Encoder and Language-Specific Decoders (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to train multiple languages with a shared encoder and multiple decoders are based on denoising autoencoding of each language and back-translating between English and multiple non-English languages. |
| Approach: | They propose a multilingual unsupervised NMT scheme which trains multiple languages with a shared encoder and multiple decoders. |
| Outcome: | The proposed model performs better than the separately trained bilingual models on monolingual corpora and improves by 1.48 BLEU points on WMT test sets. |
Lattice-Based Transformer Encoder for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) takes deterministic sequences for source representations. However, word-level or subword-level segmentation has multiple choices to split a source sequence with different word segmentors or different subword vocabulary sizes. |
| Approach: | They propose lattice-based encoders to explore effective word or subword representations in an automatic way during training. |
| Outcome: | The proposed encoders can explore effective word or subword representation in an automatic way during training. |
Collective Wisdom: Improving Low-resource Neural Machine Translation using Adaptive Knowledge Distillation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to train high-quality NMT models in bilingually low-resource scenarios are limited by the scarcity of parallel sentence-pairs. |
| Approach: | They propose to distill the knowledge of teacher models to a single student model by using knowledge distillation. |
| Outcome: | The proposed approach achieves up to +0.9 BLEU score improvements compared to strong baselines. |
Token-wise Curriculum Learning for Neural Machine Translation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing curriculum learning approaches to Neural Machine Translation (NMT) require sampling sufficient amounts of “easy” samples from training data at the early stage of training. |
| Approach: | They propose a token-wise curriculum learning approach that creates sufficient amounts of easy samples from training data. |
| Outcome: | The proposed approach outperforms baselines on five language pairs on low-resource languages. |
Continual Learning for Neural Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Neural machine translation models are data-driven and require large-scale training corpus . continual learning remains a big challenge for artificial intelligence systems and models . |
| Approach: | They propose a continual learning framework for NMT models that incorporates multiple stages of training to alleviate catastrophic forgetting problem. |
| Outcome: | The proposed framework achieves superior performance compared to baseline models in all settings. |
Neural Machine Translation of Text from Non-Native Speakers (N19-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) systems are known to degrade when confronted with noisy data. |
| Approach: | They propose to augment training data with sentences containing artificially-introduced grammatical errors to make the system more robust to such errors. |
| Outcome: | The proposed approach recovers 1.0 BLEU out of 2.4 BLUE lost due to grammatical errors on a set of Spanish translations of the JFLEG grammar error correction corpus. |
Transforming Term Extraction: Transformer-Based Approaches to Multilingual Term Extraction Across Domains (2021.findings-acl)
Copied to clipboard
| Challenge: | Automated Term Extraction (ATE) is a challenging task, with few exceptions. |
| Approach: | They propose to use a transformer-based term extraction model to extract terms from sentences . they also propose to employ a language model for token classification and a sequence model to reduce sentences to terms . |
| Outcome: | The proposed models outperform baselines on the ATE challenge TermEval 2020 dataset in English, French, and Dutch. |
Generating Authentic Adversarial Examples beyond Meaning-preserving with Doubly Round-trip Translation (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to generate adversarial examples for NMT use the meaning-preserving restriction. |
| Approach: | They propose a new definition for adversarial examples based on the Doubly Round-Trip Translation (DRTT) they introduce masked language models to construct bilingual adversarials based upon DRTT . |
| Outcome: | The proposed approach significantly improves the robustness of the NMT model on clean and noisy test sets. |
Dual-teacher Knowledge Distillation for Low-frequency Word Translation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Neural machine translation models are trained on parallel corpora with unbalanced word frequency distribution, resulting in high-frequency words being ignored. |
| Approach: | They propose to employ a low-frequency teacher model that excels in translating low- frequency words to guide the learning of the student model. |
| Outcome: | The proposed method achieves +0.64 BLEU improvements over the state-of-the-art method on the low-frequency translation task while maintaining the translation quality of high-frequency words. |
A Retrieve-and-Rewrite Initialization Method for Unsupervised Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Recent work shows successful methods for unsupervised machine translation (UMT) initialization stage is important since bad initialization may wrongly squeeze the search space and too much noise may hurt the final performance. |
| Approach: | They propose a retrieval and rewriting based method to better initialize unsupervised translation models. |
| Outcome: | The proposed method improves translation performance by over 4 BLEU scores. |
Does Multi-Encoder Help? A Case Study on Context-Aware Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | In encoder-decoder neural models, multiple encoders are used to represent contextual information in addition to the individual sentence. |
| Approach: | They propose to use multiple context encoders to encode the individual sentences in document-level neural machine translation (NMT) They propose a noisy dropout setup and a single-encoder approach to encode context sentences. |
| Outcome: | The proposed approach encodes the context and the current sentence without contexts. |
Revisiting Robust Neural Machine Translation: A Transformer Case Study (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in NMT have shown promising results but are vulnerable to noise. |
| Approach: | They propose a data-driven technique called Target Augmented Fine-tuning to incorporate noise during training. |
| Outcome: | The proposed techniques perform with no degradation where up to 10% of entire test words are infected by noise. |
Benchmarking Neural and Statistical Machine Translation on Low-Resource African Languages (2020.lrec-1)
Copied to clipboard
| Challenge: | a recent study has focused on languages where large amounts of resources are available. |
| Approach: | They benchmark state of the art statistical and neural machine translation systems on Somali and Swahili languages . they find that statistical machine translation and neural translation can perform similarly in low-resource scenarios . |
| Outcome: | The results show that statistical machine translation and neural machine translation perform similarly in low-resource scenarios. |
Can Synthetic Translations Improve Bitext Quality? (2022.acl-long)
Copied to clipboard
| Challenge: | Synthetic translations have been used for a wide range of NLP tasks, but it remains unclear how they differ from naturally occurring data. |
| Approach: | They propose to use a semantic equivalence classifier to improve bitext quality without additional bilingual supervision to replace the originals. |
| Outcome: | The proposed samples improve bitext quality without additional bilingual supervision and are validated intrinsically and extrinsically through bilingual induction and MT tasks. |
On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Neural machine translation suffers from exposure bias, and alternative approaches to mitigate this are under debate. |
| Approach: | They propose to reduce exposure bias by using minimum risk training to mitigate hallucinations . they find that exposure bias is more problematic under domain shift . |
| Outcome: | The proposed methods can reduce exposure bias even on in-domain test sets. |
SrcMix: Mixing of Related Source Languages Benefits Extremely Low-resource Machine Translation (2026.findings-eacl)
Copied to clipboard
| Challenge: | Multilingual models are widely used for machine translation, but their effectiveness for extremely low-resource languages (ELRLs) is dependent on how related languages are incorporated during fine-tuning. |
| Approach: | They propose a source-side mixing strategy that combines related ELRLs during fine-tuning while constraining the decoder to a single target language. |
| Outcome: | The proposed approach improves performance in high-resource to ELRL translations and in mid-resourced to MT translations. |
Improving Neural Machine Translation by Multi-Knowledge Integration with Prompting (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods focus on how to integrate multiple types of knowledge into NMT models . |
| Approach: | They propose a framework that integrates multiple types of knowledge into NMT models . they use multiple types as prefix-prompts of input for the encoder and decoder . |
| Outcome: | The proposed framework outperforms baselines on English-Chinese and English-German translation. |
Getting Gender Right in Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | linguistics studies show that the language used by males and females differs in terms of style and syntax. |
| Approach: | They integrate gender information into NMT systems to improve translation quality for multiple language pairs by incorporating gender information to a large dataset. |
| Outcome: | The proposed system significantly improves translation quality for some language pairs. |
Training Deeper Neural Machine Translation Models with Transparent Attention (D18-1)
Copied to clipboard
| Challenge: | Existing NMT models are shallow in comparison to convolutional models used for both text and vision tasks. |
| Approach: | They propose to modify the attention mechanism to ease the optimization of deeper models by a simple modification to the seq2seq with attention paradigm. |
| Outcome: | The proposed model achieves consistent gains of 0.7-1.1 BLEU on the benchmark WMT’14 English-German and WMT'15 Czech-English tasks. |
Encoding Gated Translation Memory into Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Neural machine translation (MT) technology has made significant progress in the past few years. |
| Approach: | They propose a method to combine the strengths of TM and neural machine translation (NMT) they use a gating mechanism to balance the impact of the TM match on the NMT decoder . |
| Outcome: | The proposed method improves translation quality by over 10 BLEU points when fuzzy matches are higher than 50% on the UN corpus. |
SimulSpeech: End-to-End Simultaneous Speech to Text Translation (2020.acl-main)
Copied to clipboard
| Challenge: | SimulSpeech is an end-to-end simultaneous speech to text translation system . conventional approaches to simultaneous speech translation divide the translation process into two stages . |
| Approach: | They develop an end-to-end simultaneous speech to text translation system which translates speech in source language to text in target language concurrently. |
| Outcome: | The proposed system achieves reasonable BLEU scores and lower delay compared to full-sentence translation model. |
On Creating an English-Thai Code-switched Machine Translation in Medical Domain (2024.findings-emnlp)
Copied to clipboard
Parinthapat Pengpun, Krittamate Tiankanon, Amrest Chinkamol, Jiramet Kinchagawat, Pitchaya Chairuengjitjaras, Pasit Supholkhan, Pubordee Aussavavirojekul, Chiraphat Boonnag, Kanyakorn Veerakanjana, Hirunkul Phimsiri, Boonthicha Sae-jia, Nattawach Sataudom, Piyalitt Ittichaiwong, Peerat Limkonchotiwat
| Challenge: | despite advances in English-Thai MT, common MT approaches often underperform in the medical field due to their inability to precisely translate medical terminologies. |
| Approach: | They propose to maintain medical terminology in English within translated text through code-switched translation. |
| Outcome: | The proposed method shows that medical professionals prefer CS translations that maintain critical English terms accurately, even if it slightly compromises fluency. |
Shared-Private Bilingual Word Embeddings for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Word embedding is central to neural machine translation, but indirectly interfaces with other layers, making them comparatively isolated. |
| Approach: | They propose a shared-private bilingual word embedding which gives a closer relationship between the source and target embedders and reduces the number of model parameters. |
| Outcome: | The proposed model improves on 5 language pairs belonging to 6 different language families and written in 5 different alphabets and significantly reduces model parameters. |
Finding the Optimal Vocabulary Size for Neural Machine Translation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Class imbalance is said to exist when one or more classes are not of approximately equal frequency in data. |
| Approach: | They cast neural machine translation (NMT) as a classification task in an autoregressive setting and examine its limitations. |
| Outcome: | The proposed model performs better on multiple languages with large data sizes with different vocabulary sizes. |
Dynamic Curriculum Learning for Low-Resource Neural Machine Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Recent work on neural machine translation (NMT) has demonstrated impressive performance improvements and became the de-facto standard. |
| Approach: | They propose a dynamic curriculum learning method to reorder training samples in training using a Transformer-based system. |
| Outcome: | The proposed method outperforms baselines on three low-resource machine translation benchmarks and different sized data of WMT’16 En-De. |
Continuous Language Generative Flow (2021.acl-long)
Copied to clipboard
| Challenge: | Recent years have witnessed various types of generative models for natural language generation (NLG), especially RNNs or transformers. |
| Approach: | They propose a flow-based language generation model that adapts flow-derived generative models to language generation via continuous input embeddings, adapted affine coupling structures, and a novel architecture for autoregressive text generation. |
| Outcome: | The proposed model improves on QG and NMT and improves performance over baselines on SQuAD and TVQA and NML16. |
Improving Machine Translation with Phrase Pair Injection and Corpus Filtering (2022.emnlp-main)
Copied to clipboard
| Challenge: | In this paper, we show that the combination of Phrase Pair Injection and Corpus Filtering boosts the performance of Neural Machine Translation systems. |
| Approach: | They propose to combine Phrase Pair Injection and Corpus Filtering to boost performance of Neural Machine Translation systems. |
| Outcome: | The proposed method improves machine translation models on low-resource language pairs . BLEU score improves over models trained with whole pseudo-parallel corpus augmented with parallel corpus. |
Jam or Cream First? Modeling Ambiguity in Neural Machine Translation with SCONES (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing neural machine translation models learn the probability P (y|x) of the target sentence given the source sentence x. |
| Approach: | They propose to replace softmax activation with a multi-label classification layer that can model ambiguity more effectively. |
| Outcome: | The proposed multi-label classification layer can model ambiguity more effectively . it yields consistent BLEU score gains across six translation directions . |
On Compositional Generalization of Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Modern neural machine translation models have shown competitive performance in benchmarks such as WMT, but there are significant issues such as robustness, domain generalization, etc. |
| Approach: | They propose a benchmark dataset for NMT models from the perspective of compositional generalization and quantitatively analyze the results. |
| Outcome: | The proposed model performs well under traditional metrics, but is low in out-of-domain and low-resource conditions. |
On the Copying Behaviors of Pre-Training for Neural Machine Translation (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that initializing NMT models with pre-trained language models (LM) can speed up the model training and boost the model performance. |
| Approach: | They propose a method to control copying behaviors in NMT models by initializing them with pre-trained language models (LM) they propose to use a metric called copy ratio to control the copying behavior in decoding. |
| Outcome: | The proposed method improves translation performance by controlling copying behaviors for pre-training based models. |
Token Drop mechanism for Neural Machine Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Neural machine translation models are vulnerable to unfamiliar inputs. |
| Approach: | They propose to drop tokens of the input sentences to improve generalization and avoid overfitting for the NMT model. |
| Outcome: | The proposed approach improves on Chinese-English and English-Romanian benchmarks and achieves significant performance improvements over baselines. |
Vocabulary Adaptation for Domain Adaptation in Neural Machine Translation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Neural network methods exhibit strong performance only in a few resource-rich domains. |
| Approach: | They propose a method that fine-tunes embedding layers of a pre-trained NMT model to the target domain. |
| Outcome: | The proposed method improves fine-tuning performance in En-Ja and De-En translation by 3.86 and 3.28 BLEU points. |
Filtering Back-Translated Data in Unsupervised Neural Machine Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Current state of the art approaches for unsupervised neural machine translation (NMT) use only monolingual data for training. |
| Approach: | They propose an approach to filter back-translated data as part of the training process of unsupervised neural machine translation (NMT) they propose a weight component based on the quality of pseudo parallel sentence pairs generated in back-translation phase. |
| Outcome: | The proposed approach improves the training performance of unsupervised neural machine translation systems by giving weight to good pseudo parallel sentence pairs in the back-translation phase. |
Unsupervised Extraction of Partial Translations for Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | Neural machine translation systems usually require a large quantity of bilingual parallel data for training. |
| Approach: | They propose an algorithm for extracting from monolingual data what they call partial translations . partial translation is a pair of source and target sentences that contain sequences of tokens that are translations of each other. |
| Outcome: | The proposed algorithm extracts from monolingual data what we call partial translations . it takes only source and target monolingual datasets as input . |
Addressing word-order Divergence in Multilingual Neural Machine Translation for extremely Low Resource Languages (N19-1)
Copied to clipboard
| Challenge: | Existing studies show that transfer learning works best when the languages are related. |
| Approach: | They propose to pre-order assisting language sentences to match the word order of the source language and train the parent model. |
| Outcome: | The proposed model can improve translation quality in low-resource scenarios by pre-ordering the assisting language sentences to match the word order of the source language and training the parent model. |
Neural Machine Translation with Phrase-Level Universal Visual Representations (2022.acl-long)
Copied to clipboard
| Challenge: | Existing multimodal machine translation methods require paired input of source sentence and image, which makes them suffer from shortage of sentence-image pairs. |
| Approach: | They propose a phrase-level retrieval-based method to get visual information from existing sentence-image data sets. |
| Outcome: | The proposed method significantly outperforms strong baselines on multiple MMT datasets, especially when the textual context is limited. |
Taking Actions Separately: A Bidirectionally-Adaptive Transfer Learning Method for Low-Resource Neural Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing approaches to train NMT models rely on sparse parallel data . a variety of PC variants yield significant improvements for low-resource NMT . |
| Approach: | They propose to transfer well-trained NMT models to low-resource languages by bidirectionally-adaptive learning strategy . they divide inner constituents of Parent encoder into two "teams" aiming to adapt to characteristics of low- and high-resourced languages . |
| Outcome: | The proposed method improves on low-resource NMT models with a variety of PC variants. |
DC-MBR: Distributional Cooling for Minimum Bayesian Risk Decoding (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for decoding target language are degenerate, hallucinating or empty. |
| Approach: | They propose a method that tunes down the Softmax temperature to reduce autoregressive over-smoothness by label smoothing the output distributions. |
| Outcome: | The proposed method improves MBR in various settings. |
Knowledge Graph Enhanced Neural Machine Translation via Multi-task Learning on Sub-entity Granularity (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods to integrate knowledge graph (KG) with neural machine translation (NMT) have two problems: knowledge under-utilization and granularity mismatch. |
| Approach: | They propose a multi-task learning method on sub-entity granularity to combine machine translation and knowledge reasoning tasks. |
| Outcome: | The proposed method significantly outperforms baseline models on translation tasks and handling the entities. |
A Study of Reinforcement Learning for Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Recent studies have shown that reinforcement learning (RL) is an effective approach for improving the performance of neural machine translation systems. |
| Approach: | They propose to leverage reinforcement learning to boost the performance of NMT systems trained with monolingual data. |
| Outcome: | The proposed method achieves competitive results on translation tasks in English-German, Chinese-English and English-English systems. |
Meta-Learning for Low-Resource Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | In this paper, we propose to extend the recently introduced model-agnostic meta-learning algorithm for low-resource neural machine translation (NMT). |
| Approach: | They propose to extend the recently introduced meta-learning algorithm for low-resource neural machine translation (NMT) they frame low-Resource translation as a meta- learning problem where we learn to adapt to low-REsource languages based on multilingual high-resourced language tasks. |
| Outcome: | The proposed meta-learning algorithm outperforms the multilingual, transfer learning based approach and can train a competitive NMT system with only a fraction of training examples. |
F-MALLOC: Feed-forward Memory Allocation for Continual Learning in Neural Machine Translation (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches to address Catastrophic Forgetting (CF) have been developed to avoid forgetting and maintain system extensibility. |
| Approach: | They propose a method to reduce Catastrophic Forgetting (CF) by decomposing feed-forward layers into discrete memory cells and ensuring robust extendability. |
| Outcome: | The proposed method achieves higher BLEU scores and almost zero forgetting while maintaining robust extendability. |
Large Language Models for Persian-English Idiom Translation (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models have shown superior capabilities in translating figurative language compared to neural machine translation systems. |
| Approach: | They evaluate LLMs, NMTs and their combinations using PersianIdioms datasets . they find that automatic evaluation methods like BLEU and BERTScore are effective . |
| Outcome: | The proposed model performs better in both directions than other models. |
Nearest Neighbor Knowledge Distillation for Neural Machine Translation (2022.naacl-main)
Copied to clipboard
| Challenge: | k-nearest-neighbor machine translation (kNN-MT) is a state-of-the-art machine translation technique . however, it requires conducting kNN searches for each decoding step, which increases the cost of decoding . |
| Approach: | They propose to move the time-consuming kNN search forward to the preprocessing phase and introduce k Nearest Neighbor Knowledge Distillation (kNN-KD) that trains the base NMT model to directly learn the knowledge of kN. |
| Outcome: | The proposed method improves over the state-of-the-art model while maintaining the same training and decoding speed as the standard model. |
Modeling Consistency Preference via Lexical Chains for Document-level Neural Machine Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that consistency preference for lexical chains reduces lexical translation inconsistency . Lexical translation consistency is a common discourse phenomenon . |
| Approach: | They propose a consistency-aware model which captures consistency context . they then define consistency-tailored latent variables which guide translation of corresponding sentences . |
| Outcome: | The proposed model significantly improves translation performance in ChineseEnglish and FrenchEnglish translation tasks. |
On the Sub-layer Functionalities of Transformer Decoder (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing efforts to interpret the encoder of Transformer-based encoder-decoder architectures for neural machine translation have focused on assessing the encoded representations or interpreting the multi-head self-attentions. |
| Approach: | They propose to use Transformer-based encoder-decoder architectures to analyze how information is propagated through each module of each decoder layer. |
| Outcome: | The proposed model can be dropped with minimal loss of performance on three translation datasets and can be used to train and inference faster. |
Breaking the Representation Bottleneck of Chinese Characters: Neural Machine Translation with Stroke Sequence Modeling (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing research treats Chinese character as a minimum unit for representation . however, such representation suffers from two bottlenecks: 1) learning bottleneck; 2) parameter bottleneck, each individual character has to be represented by a unique vector. |
| Approach: | They propose a representation method for Chinese characters to break the representation bottlenecks . they map each stroke to a specific Latin character, thus allowing similar Chinese characters . |
| Outcome: | The proposed representation method breaks two representation bottlenecks in Chinese character representation . it maps each stroke to a specific Latin character, thus allowing similar Chinese characters to have similar representations . |
Universal Conditional Masked Language Pre-training for Neural Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Pre-trained sequence-to-sequence models have significantly improved Neural Machine Translation (NMT) this paper demonstrates that pre-training a sequence- to-squence model with a bidirectional decoder can produce notable performance gains for both Autoregressive and Non-autoregressive NMT tasks. |
| Approach: | They propose a conditional masked language model pre-trained on bilingual and monolingual corpora in many languages. |
| Outcome: | The proposed model can achieve significant performance improvements on all scenarios from low- to extremely high-resource languages. |
Neural Machine Translation for Low-Resourced Indian Languages (2020.lrec-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is an effective way to convert text to a different language without human involvement. |
| Approach: | They propose to use multihead self-attention along with pre-trained Byte-Pair-Encoded (BPE) and MultiBPE embeddings to develop an efficient machine translation system. |
| Outcome: | The proposed system outperforms Google translator and the existing translators on two of the most morphological rich Indian languages. |
Modeling Bilingual Conversational Characteristics for Neural Chat Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Neural chat translation aims to translate bilingual conversational text due to its inherent characteristics such as role preference, dialogue coherence, and translation consistency. |
| Approach: | They propose to model the translation quality of conversational text by learning distributions of bilingual conversational characteristics. |
| Outcome: | The proposed approach outperforms baseline models and is widely available. |
Multi-level Community-awareness Graph Neural Networks for Neural Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Recent studies have used Graph Neural Networks (GNNs) to encode language knowledge into token embeddings. |
| Approach: | They propose a multi-level community-awareness Graph Neural Network layer to jointly model local and global relationships between words and their linguistic roles in multiple communities. |
| Outcome: | The proposed method reduces time complexity in very long sentences while preserving the original meaning. |
Content-Equivalent Translated Parallel News Corpus and Extension of Domain Adaptation for NMT (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods to train NMT systems with noisy data are not sufficient . a recent increase in foreigners visiting Japan has created a significant information gap . |
| Approach: | They propose a Japanese-English parallel news corpus that is content-equivalent . they extend a domain-adaptation method to train NMT models with clean corpus . |
| Outcome: | The proposed corpus improves translation quality and is more effective than existing methods. |
On the Complementarity between Pre-Training and Random-Initialization for Resource-Rich Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Pre-Training (PT) of text representations has been successfully applied to low-resource Neural Machine Translation (NMT) however, it often fails to achieve notable gains on resource-rich NMT on par with its Random-Initialization (RI) counterpart. |
| Approach: | They propose to combine pre-training and random-initialization techniques to achieve significant improvements in NMT. |
| Outcome: | The proposed model fusion algorithm can achieve significant improvements on two resource-rich translation benchmarks. |
NMT and PBSMT Error Analyses in English to Brazilian Portuguese Automatic Translations (2020.lrec-1)
Copied to clipboard
| Challenge: | Recent work proposes neural machine translation (NMT) for Brazilian Portuguese. |
| Approach: | They propose a neural machine translation approach that generates equivalent sentences in target language and source language. |
| Outcome: | The proposed approach outperforms phrase-based statistical machine translation systems for some pairs of languages. |
Iterative Constrained Back-Translation for Unsupervised Domain Adaptation of Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing back-translation methods focus on in-domain lexical knowledge, which may lead to poor translation of unseen in- domain words. |
| Approach: | They propose an iterative constrained back-translation method to incorporate in-domain lexical knowledge into synthetic parallel data from BT. |
| Outcome: | The proposed method improves the BLEU score by up to 3.08 on four domains. |
Linguistically-Motivated Yorùbá-English Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Several phenomena where asymmetry arises have been identified as challenging problems for machine translation. |
| Approach: | They perform a fine-grained analysis of how an SMT system compares with two NMT systems when translating bare nouns into English. |
| Outcome: | The proposed model outperforms the SMT and BiLSTM models for 4 categories and the BiLST outperformed the SLT models for 3 categories. |
Being Generous with Sub-Words towards Small NMT Children (2020.lrec-1)
Copied to clipboard
| Challenge: | In the context of under-resourced neural machine translation, transfer learning from an NMT model trained on a high resource language pair, or from a multilingual NMT (M-NMT) model, has been shown to boost performance to a large extent. |
| Approach: | They propose to use a multilingual NMT model to train on an under-resourced child and to use large sub-word vocabularies to improve performance. |
| Outcome: | The proposed approach involving dynamic vocabularies is both practical and effective on two under-resourced language pairs, i.e. Icelandic-English and Irish-English. |
Dynamic Position Encoding for Transformers (2022.coling-1)
Copied to clipboard
| Challenge: | In neural machine translation, the general task of translating is to reduce the input sentence into smaller units (also known as statistical phrases), select an optimal translation for each unit, and place them in the correct order. |
| Approach: | They propose a novel architecture that relies on a feed-forward backbone and self-attention mechanism to encode sequential/positional information. |
| Outcome: | The proposed architecture improves on multiple datasets in French, Italian, and German and shows that it is more efficient than the current model. |
PAEG: Phrase-level Adversarial Example Generation for Neural Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for adversarial example generation are word-level or character-level, which ignore the ubiquitous phrase structure. |
| Approach: | They propose a phrase-level adversarial example generation framework to enhance the robustness of the translation model by adopting a sentence-level substitution strategy. |
| Outcome: | The proposed method improves translation performance and robustness to noise on three benchmarks. |
JASS: Japanese-specific Sequence to Sequence Pre-training for Neural Machine Translation (2020.lrec-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) requires large parallel corpora for training robust and high quality models. |
| Approach: | They propose a Japanese-specific sequence to sequence pre-training alternative to MASS for NMT . they use Japanese as the source or target language to train their models . |
| Outcome: | The proposed approach can give competitive results over MASS and BRSS, and significantly surpass the individual methods. |
Addressing Asymmetry in Multilingual Neural Machine Translation with Fuzzy Task Clustering (2022.coling-1)
Copied to clipboard
| Challenge: | Existing clustering methods cannot handle asymmetric problem in multilingual NMT . existing models cannot handle the asymmetry problem since there are thousands of languages involved . |
| Approach: | They propose a fuzzy task clustering method to address the asymmetric problem in multilingual NMT by using task affinity as the clustering criterion. |
| Outcome: | The proposed method outperforms baselines for a multilingual model and the existing models. |
MuST-Cinema: a Speech-to-Subtitles corpus (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for subtitling are laborious and costly, says aaron sanchez . he says the current methods are laboriously complex and require manual work . |
| Approach: | They propose to use TED subtitles to build a multilingual speech translation corpus . they propose to annotate existing subtitling corpora with subtitle breaks . |
| Outcome: | The proposed model can be used to segment sentences into subtitles and reduces human work . the proposed model reduces the time and cost of human subtitling tasks . |
Improving Both Domain Robustness and Domain Adaptability in Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing approaches to domain adaptation for NMT depend on high-quality parallel data. |
| Approach: | They propose a meta-learning framework which improves domain robustness and adaptability . they use a word-level domain mixing model and a domain classifier to integrate it . |
| Outcome: | The proposed approach improves domain robustness and adaptability in seen and unseen domains. |
ParaPat: The Multi-Million Sentences Parallel Corpus of Patents Abstracts (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing parallel corpora for patents and scientific texts are not available due to the need for correct alignment and human curation. |
| Approach: | They develop a parallel corpus from the open access Google Patents dataset . they use Hunalign algorithm to align sentences and tokens using the largest 22 languages . |
| Outcome: | The proposed corpus is available in TSV format and with a SQLite database, with complementary information regarding patent metadata. |
Faster Machine Translation Ensembling with Reinforcement Learning and Competitive Correction (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent approaches to ensembling neural machine translation models require inference across all candidate models, leading to significant computational overhead. |
| Approach: | They propose a reinforcement learning-based strategy that improves the CSB by selecting a small, fixed number of candidates and identifying optimal groups to pass to the fusion block for each input sentence. |
| Outcome: | The proposed approach improves the CSB by selecting a small, fixed number of candidates and identifying optimal groups to pass to the fusion block for each input sentence. |
Towards Robust Neural Machine Translation with Iterative Scheduled Data-Switch Training (2022.coling-1)
Copied to clipboard
Zhongjian Miao, Xiang Li, Liyan Kang, Wen Zhang, Chulun Zhou, Yidong Chen, Bin Wang, Min Zhang, Jinsong Su
| Challenge: | Existing methods on robust neural machine translation (NMT) construct adversarial examples by injecting noise into authentic examples and indiscriminately exploit two types of examples. |
| Approach: | They propose an iterative scheduled data-switch training framework to mitigate this problem by injecting noise into authentic examples and indiscriminately exploiting two types of examples. |
| Outcome: | The proposed model outperforms several competitive benchmarks on four translation benchmarks. |
Multiword Expression aware Neural Machine Translation (2020.lrec-1)
Copied to clipboard
| Challenge: | Multiword Expressions (MWEs) are a pervasive phenomenon in all natural languages and challenge NLP applications because of their unpredictable morpho-syntactic and lexico--semantic behaviour. |
| Approach: | They propose to use linguistic resources to improve MWE translation and MWE generation by up to 5.09 BLEU points on MWE test sets. |
| Outcome: | The proposed annotation and data augmentation improve translation quality and increase performance by up to 5.09 BLEU points on MWE test sets. |
From Priest to Doctor: Domain Adaptation for Low-Resource Neural Machine Translation (2025.coling-main)
Copied to clipboard
| Challenge: | Existing data for low-resource languages are limited; the languages that could most benefit from domain adaptation (DA) are the ones left behind. |
| Approach: | They propose a realistic setting in which they aim to translate between a high-resource and a low-resourced language with limited parallel data, a bilingual dictionary, and c) a monolingual target-domain corpus in the high-rsource language. |
| Outcome: | The proposed methods are compared with a human evaluation of DALI and show that the most effective is the simplest. |
A Large-Scale Study of Machine Translation in Turkic Languages (2021.emnlp-main)
Copied to clipboard
Jamshidbek Mirzakhalov, Anoop Babu, Duygu Ataman, Sherzod Kariev, Francis Tyers, Otabek Abduraufov, Mammad Hajili, Sardana Ivanova, Abror Khaytbaev, Antonio Laverghetta Jr., Bekhzodbek Moydinboyev, Esra Onal, Shaxnoza Pulatova, Ahsan Wahab, Orhan Firat, Sriram Chellappan
| Challenge: | a large corpus covering 22 Turkic languages is included in this paper . low-resource MT evaluation has traditionally focused on European languages due to limitations of available technology and resources. |
| Approach: | They present a case study of the practical application of MT in the Turkic language family . they propose to realize the gains of NMT for Turkic languages under high-resource to extremely low-resourced scenarios. |
| Outcome: | The proposed study shows that the new methods can be used in the Turkic language family . the results highlight bottlenecks in building competitive systems . |
Hypoformer: Hybrid Decomposition Transformer for Edge-friendly Neural Machine Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to compress Transformer are limited to sub-components, e.g., selfattention networks or embedding layer. |
| Approach: | They propose a Hybrid Tensor-Train decomposition which retains full rank and meanwhile reduces operations and parameters. |
| Outcome: | The proposed model outperforms light-weight SOTA methods on three translation tasks and achieves 7.1 points absolute improvement in BLEU and 1.27 X speedup on IWSLT’14 De-En task. |
Rule-based Morphological Inflection Improves Neural Terminology Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Current approaches to incorporating terminology constraints in machine translation (MT) typically assume that the constraint terms are provided in their correct morphological forms. |
| Approach: | They propose a framework for incorporating lemma constraints in machine translation . they use a cross-lingual inflection module that inflects the target lemmo constraints based on the source context. |
| Outcome: | The proposed framework outperforms existing methods with lower training costs and linguistic knowledge in domain adaptation and low-resource MT settings. |
Integrating Vectorized Lexical Constraints for Neural Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing studies focus on integrating discrete lexical constraints into neural machine translation models. |
| Approach: | They propose to integrate constraints into NMT models by integrating them into keys and values . they show that their method outperforms representative baselines on four language pairs . |
| Outcome: | The proposed method outperforms baselines on four language pairs, showing superiority . |
Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translation (2025.findings-acl)
Copied to clipboard
Yingfeng Luo, Tong Zheng, Yongyu Mu, Bei Li, Qinghong Zhang, Yongqi Gao, Ziqiang Xu, Peinan Feng, Xiaoqian Liu, Tong Xiao, JingBo Zhu
| Challenge: | Recent advances in machine translation have focused on a single pre-trained decoder . encoder-decoder architectures have received relatively little attention in NMT . |
| Approach: | They propose a method that leverages LLMs as MT encoders and pairs them with lightweight decoders to develop universal translation models. |
| Outcome: | The proposed method matches or surpasses baselines in terms of translation quality but achieves 75% reduction in memory footprint of the KV cache. |
Multilingual Data Filtering using Synthetic Data from Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have shown that effective filters can be created by utilising Large Language Models to synthetically label data, which is then used to train smaller neural models for filtering purposes. |
| Approach: | They extend this approach to languages beyond English to train neural models for filtering purposes. |
| Outcome: | The proposed approach is effective at filtering parallel text for translation quality and filtering for domain specificity. |
Selective Knowledge Distillation for Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Neural Machine Translation models achieve state-of-the-art performance on many translation benchmarks. |
| Approach: | They propose a protocol that analyzes different impacts of samples by comparing various samples’ partitions. |
| Outcome: | The proposed methods yield up to +1.28 and +0.89 BLEU points improvements over the Transformer baseline, respectively. |
A Tree-based Decoder for Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Existing work on adding syntactic information to NMT systems is limited to linguistically-inspired tree structures. |
| Approach: | They propose an NMT model that can naturally generate the topology of an arbitrary tree structure on the target side. |
| Outcome: | The proposed model outperforms standard seq2seq models by 2.1 BLEU points and other methods for incorporating target-side syntax by 0.7 BLUE points. |
Synthetic Pre-Training Tasks for Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | toxicity and bias can be addressed by pre-training with synthetic resources . BLEU scores are used to compare methods with real-world data . |
| Approach: | They propose several ways to generate obfuscated data from large parallel corpus and concatenating phrase pairs from small word-aligned corpus with synthetic parallel data without real human language corpora. |
| Outcome: | The proposed methods can be used to generate obfuscated data or synthetic parallel data without real human language corpora even with high levels of oblication. |
Memorisation Cartography: Mapping out the Memorisation-Generalisation Continuum in Neural Machine Translation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Using the counterfactual memorisation metric, we find that when training neural networks, models will memorise some inputs but not others. |
| Approach: | They use the counterfactual memorisation metric to build a resource that places 5M NMT datapoints on a memorisations-generalisation map and describe how the datapoint’s surface-level characteristics and a models’ per-datum training signals are predictive of memorising in NMT. |
| Outcome: | The proposed model places 5M NMT datapoints on a memorisation-generalisation map and shows how their surface-level characteristics and models’ per-datum training signals are predictive of memorising in NMT. |
MTCue: Learning Zero-Shot Control of Extra-Textual Attributes by Leveraging Unstructured Context in Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing research has focused on providing individual, well-defined types of context in translation, such as the surrounding text or discrete external variables like the speaker’s gender. |
| Approach: | They introduce a novel neural machine translation framework that interprets all context as text. |
| Outcome: | The proposed framework outperforms a baseline that matched the parameters and significantly outperformed it in English translation. |
Prediction Difference Regularization against Perturbation for Neural Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for regularizing input perturbation are limited by under-fitting of training data. |
| Approach: | They propose a method that can reduce over-fitting and under-fitting at the same time. |
| Outcome: | The proposed method can reduce over-fitting and under-fitturing while making the model less sensitive to small input changes and more robust to under-perturbed training data. |
AdvAug: Robust Adversarial Augmentation for Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Recent work in neural machine translation has led to dramatic improvements in both research and commercial systems. |
| Approach: | They propose a adversarial augmentation method for Neural Machine Translation that minimizes vicinal risk over virtual sentences . they use a novel vicinity distribution for adversarials to describe a smooth interpolated embedding space . |
| Outcome: | The proposed method outperforms the current method on Chinese-English, English-French, and English-German translation benchmarks. |
Improving Neural Machine Translation with Soft Template Prediction (2020.acl-main)
Copied to clipboard
| Challenge: | Recent advances in neural machine translation (NMT) depend on source text to generate translation. |
| Approach: | They propose to use extracted templates from tree structures as soft target templates to guide the translation procedure. |
| Outcome: | The proposed model outperforms baseline models on four benchmarks and demonstrates the effectiveness of soft target templates. |
DivEMT: Neural Machine Translation Post-Editing Effort Across Typologically Diverse Languages (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in neural language modeling and multilingual training have prompted widespread adoption of machine translation (MT) technologies across an unprecedented range of world languages. |
| Approach: | They propose to use a dataset to assess the impact of two state-of-the-art NMT systems, Google Translate and the multilingual mBART-50 model, on translation productivity. |
| Outcome: | The proposed model is faster than translation from scratch, but the magnitude of productivity gains varies widely across systems and languages. |
BERT, mBERT, or BiBERT? A Study on Contextualized Embeddings for Neural Machine Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for incorporating pre-trained models into NMT systems are non-trivial and lack a comparison of the impact that other pre-trainers may have on translation performance. |
| Approach: | They propose to use the input of a bilingual pre-trained language model as the input for NMT encoders and a stochastic layer selection approach to ensure sufficient utilization of contextualized embeddings. |
| Outcome: | The proposed bilingual pre-trained language model outperforms all other pre-train models on the IWSLT’14 dataset and the proposed dual-directional translation model. |
BigVideo: A Large-scale Video Subtitle Translation Dataset for Multimodal Machine Translation (2023.findings-acl)
Copied to clipboard
Liyan Kang, Luyang Huang, Ningxin Peng, Peihao Zhu, Zewei Sun, Shanbo Cheng, Mingxuan Wang, Degen Huang, Jinsong Su
| Challenge: | Existing datasets focus on captions describing images or videos, which are not large and diverse enough. |
| Approach: | They propose a large-scale video subtitle translation dataset to facilitate multi-modality machine translation. |
| Outcome: | The proposed dataset is 10 times larger than the widely used *How2* and *VaTeX* datasets. |
POMP: Probability-driven Meta-graph Prompter for LLMs in Low-resource Unsupervised Neural Machine Translation (2024.acl-long)
Copied to clipboard
| Challenge: | Low-resource languages (LRLs) face challenges in supervised neural machine translation due to limited parallel data. |
| Approach: | They propose a method that uses a dynamic graph to organize auxiliary languages in prompts to improve LRL translations. |
| Outcome: | The proposed method improves translation accuracy in low-resource languages (LRLs) using auxiliary language pairs and synthetic pseudo-parallel data. |
Improving NMT Models by Retrofitting Quality Estimators into Trainable Energy Loss (2025.coling-main)
Copied to clipboard
| Challenge: | Reinforcement learning has shown great promise in aligning language models with human preferences in a variety of text generation tasks, including machine translation. |
| Approach: | They propose a method that employs quality estimators as trainable loss networks to backpropagate to the NMT model. |
| Outcome: | The proposed method outperforms strong baselines and proximal policy optimizations on English-to-Mongolian translation. |
Depth Growing for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Neural machine translation models with tens and even more than a hundred blocks have shown effectiveness in image recognition. |
| Approach: | They propose a two-stage approach with three specially designed components to construct deeper NMT models. |
| Outcome: | The proposed approach improves on WMT14 EnglishGerman and EnglishFrench translation tasks. |
ConsistTL: Modeling Consistency in Transfer Learning for Low-Resource Neural Machine Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing transfer learning methods for low-resource NMT are static, which simply transfer knowledge from a parent model to a child model once via parameter initialization. |
| Approach: | They propose a transfer learning method that can continuously transfer knowledge from the parent model during the training of the child model. |
| Outcome: | The proposed method can transfer knowledge from the parent model to the child model during the training of the child. |
Imagination and Contemplation: A Balanced Framework for Semantic-Augmented Multimodal Machine Translation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Multimodal Machine Translation (MMT) is effective in resolving linguistic ambiguities, but visual information often introduces redundancy or noise, potentially impairing translation quality. |
| Approach: | They propose a semantic-augmented framework that integrates "Imagination" and "Contemplation" they first generate synthetic images from source text and align them with authentic images via an optimal transport loss . |
| Outcome: | The proposed framework outperforms baselines on translation datasets with visually ambiguous or weakly correlated content. |
AutoMoE: Heterogeneous Mixture-of-Experts with Adaptive Computation for Efficient Neural Machine Translation (2023.findings-acl)
Copied to clipboard
Ganesh Jawahar, Subhabrata Mukherjee, Xiaodong Liu, Young Jin Kim, Muhammad Abdul-Mageed, Laks Lakshmanan, V.S., Ahmed Hassan Awadallah, Sebastien Bubeck, Jianfeng Gao
| Challenge: | Existing MoE designs do not consider computational constraints (e.g., FLOPs, latency) Existing works in MoE consider homogeneous design where the same number of experts of the same size are placed uniformly throughout the network. |
| Approach: | They propose a framework for designing heterogeneous MoEs under computational constraints. |
| Outcome: | The proposed framework achieves 4x inference speedup and FLOPs reduction over manual models and within 1 BLEU point of MoE SwitchTransformer over benchmark datasets for NMT. |
Better OOV Translation with Bilingual Terminology Mining (P19-1)
Copied to clipboard
| Challenge: | Unseen words, also called out-of-vocabulary words, are difficult for machine translation . byte-pair encoding can be used to represent OOVs, but they are often incorrectly translated . |
| Approach: | They propose to use monolingual data to improve the translation of unseen words . they use five target language words to mine target-language sentences . |
| Outcome: | The proposed system can be used to improve translation of out-of-vocabulary words (OOVs) the proposed system is trained on Europarl and can be fine-tuned to improve the translation quality. |
Much Ado About Nothing – Identification of Zero Copulas in Hungarian Using an NMT Model (2020.lrec-1)
Copied to clipboard
| Challenge: | Zero copulas are the phenomenon that nominal predicates lack an explicit verbal copule in default present tense 3rd person indicative cases. |
| Approach: | They propose a tool that can identify and mark the location of zero copulas in Hungarian clauses that contain nominal predicates at the right position. |
| Outcome: | The proposed tool can identify and mark the location of zero copulas, i.e. where an overt copulan would appear in the non-default cases. |
Training and Adapting Multilingual NMT for Less-resourced and Morphologically Rich Languages (L18-1)
Copied to clipboard
| Challenge: | Using multilingual and multi-way neural machine translation approaches is a major advantage . training NMT systems for individual language pairs takes significantly more time than training of SMT systems . |
| Approach: | They propose to employ multilingual and multi-way neural machine translation approaches for morphologically rich languages such as Estonian and Russian. |
| Outcome: | The proposed approach improves translation quality by +3.27 BLEU points over baseline models. |
Towards Opening the Black Box of Neural Machine Translation: Source and Target Interpretations of the Transformer (2022.emnlp-main)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) relies on source sentence and target prefix attributions for each input token. |
| Approach: | They propose an interpretability method that tracks input tokens’ attributions for both contexts and extends it to any encoder-decoder Transformer-based model. |
| Outcome: | The proposed method can be extended to any encoder-decoder Transformer-based model and provides insights into their behaviour. |
Reducing Word Omission Errors in Neural Machine Translation: A Contrastive Learning Approach (P19-1)
Copied to clipboard
| Challenge: | Existing methods for reducing word omission errors in neural machine translation are prone to omit essential words on the source side. |
| Approach: | They propose a contrastive learning approach to reduce word omission errors in NMT by omitting words. |
| Outcome: | The proposed approach achieves better translation performance than baseline methods on Chinese-to-English, German-to English, and Russian-toEnglish translation tasks. |
WebNLG-IT: Construction of an aligned RDF-Italian corpus through Machine Translation techniques (2025.findings-acl)
Copied to clipboard
| Challenge: | Using NMT and hand-written rules, we created the first aligned Italian RDF-to-text corpus . |
| Approach: | They propose to use NMT to create an Italian version of the WebNLG corpus and to refine and improve the quality of the produced resource. |
| Outcome: | The proposed system is the best on the original English version and the best in the second step, it improves and refines the quality of the produced resource. |
A Multi-Task Architecture on Relevance-based Neural Query Translation (P19-1)
Copied to clipboard
| Challenge: | Existing models for cross-lingual information retrieval are not aware of the vocabulary distribution of the retrieval corpus. |
| Approach: | They propose a multi-task learning approach to train a Neural Machine Translation model with a Relevance-based Auxiliary Task (RAT) for search query translation. |
| Outcome: | The proposed model achieves 16% improvement over a strong baseline on Italian-English query-document dataset. |
Vector-Vector-Matrix Architecture: A Novel Hardware-Aware Framework for Low-Latency Inference in NLP Applications (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to improve accuracy of neural networks are slow due to computational complexity. |
| Approach: | They propose a vector-vector-matrix architecture which greatly reduces latency at inference time for NLP applications by a factor of four. |
| Outcome: | The proposed framework reduces the latency of sequence-to-sequence and Transformer models used for NMT by a factor of four. |
DictDis: Dictionary Constrained Disambiguation for Improved NMT (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to domain-specific neural machine translation (NMT) are lexically constrained and draw from domain- specific dictionaries. |
| Approach: | They propose a lexically constrained neural machine translation system that disambiguates between multiple dictionary candidates. |
| Outcome: | The proposed system disambiguates between multiple candidate translations derived from dictionaries on English-Hindi, English-German, and English-French datasets. |
Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMT (2021.emnlp-main)
Copied to clipboard
| Challenge: | Statistical MT decomposes the translation task into distinct components that are learned separately. |
| Approach: | They show that neural machine translation models acquire different competences over the course of training . previous work shows how to improve some of the competences in NMT by using lexical translation probabilities, phrase memories, alignment information. |
| Outcome: | The proposed model improves translation quality and word-by-word translation, while learning complex reordering patterns. |
Efficient Inference for Multilingual Neural Machine Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual NMT is an attractive solution for production, but to match bilingual quality, it comes at the cost of larger and slower models. |
| Approach: | They propose to use a shallow decoder with vocabulary filtering to speed up inference . they validate their findings with BLEU and chrF on 380 language pairs . |
| Outcome: | The proposed approach can be used in two 20-language multi-parallel settings. |
The Learnability of the Annotated Input in NMT Replicating (Vanmassenhove and Way, 2018) with OpenNMT (2020.lrec-1)
Copied to clipboard
| Challenge: | reproducibility of experiments is a key issue in Neural Networks, which are fed with variable samples of training data. |
| Approach: | They reproduce some of the experiments related to neural network training for Machine Translation as reported in . they annotated a sample from the EN-FR and EN-DE Europarl with syntactic and semantic annotations to train neural networks with the Nematus Neural Machine Translation toolkit. |
| Outcome: | The results obtained were lower than the original paper, but on a more limited set of annotations. |
Using Context in Neural Machine Translation Training Objectives (2020.acl-main)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) training is based on document-level metrics, not sentence-level BLEU. |
| Approach: | They propose to merge document-level metrics with batch-level documents to improve NMT training. |
| Outcome: | The proposed training is more robust for document-level metrics than sequence MRT and maximum-likelihood training. |
Variational Neural Machine Translation with Normalizing Flows (2020.acl-main)
Copied to clipboard
| Challenge: | Existing frameworks for learning informative latent variables are limited by limitations . existing models rely on strong assumptions on distribution of latent code . |
| Approach: | They propose to apply a variational neural machine translation framework to a Transformer . they propose to introduce a more flexible approximate posterior based on normalizing flows . |
| Outcome: | The proposed framework outperforms baseline models under in-domain and out-of-domain conditions. |
Neural Machine Translation Methods for Translating Text to Sign Language Glosses (2023.acl-long)
Copied to clipboard
| Challenge: | State-of-the-art techniques common to low resource Machine Translation (MT) are applied to improve MT of spoken language text to Sign Language glosses. |
| Approach: | They propose to use data augmentation, semi-supervised Neural Machine Translation, transfer learning and multilingual NMT to improve MT of spoken language to Sign Language glosses. |
| Outcome: | The proposed models outperform previous work on two German SL corpora and are confirmed by human evaluation. |
Domain adapted machine translation: What does catastrophic forgetting forget and why? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) models can be specialized by domain adaptation, often fine-tuning on a dataset of interest. |
| Approach: | They propose a novel approach to understanding catastrophic forgetting during NMT adaptation by investigating the relationship between the data and the in-domain vocabulary coverage. |
| Outcome: | The proposed model can be specialized by fine-tuning on a domain of interest, but can fail to achieve the predicted quality of the target domain. |
Low-resource Neural Machine Translation: Benchmarking State-of-the-art Transformer for Wolof<->French (2022.lrec-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) systems can translate between French (FR) 1 and Wolof (WO, ISO 639-3), a lowresource Niger-Congo language mainly spoken in Senegal (Gamble, 1950). |
| Approach: | They propose two neural machine translation systems based on sequence-to-sequence with attention and Transformer architectures to translate between French (FR) 1 and Wolof (WO, ISO 639-3). |
| Outcome: | The proposed models outperform the classic sequence-to-sequence model in all settings and are less sensitive to noise. |
Learning How to Translate North Korean through South Korean (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing NLP systems cannot properly handle North Korean inputs, despite limited data . Several NLP researchers have been working on the Korean language . |
| Approach: | They propose to manually create evaluation data for automatic alignment and machine translation, and investigate automatic alignment methods suitable for North Korean. |
| Outcome: | The proposed model trained by North Korean bilingual data significantly boosts translation accuracy compared to existing South Korean models in zero-shot settings. |
FGraDA: A Dataset and Benchmark for Fine-Grained Domain Adaptation in Machine Translation (2022.lrec-1)
Copied to clipboard
Wenhao Zhu, Shujian Huang, Tong Pu, Pingxuan Huang, Xu Zhang, Jian Yu, Wei Chen, Yanfeng Wang, Jiajun Chen
| Challenge: | Recent research on domain adaptation neglects diversity in translation within a domain . current research on NMT models considers very broad target domains . |
| Approach: | They propose a fine-grained domain adaptation task for autonomous vehicles, AI education, real-time networks, and smart phone. |
| Outcome: | The proposed task is compared with a dataset of Chinese-English translation tasks for four sub-domains of information technology: autonomous vehicles, AI education, real-time networks, and smart phone. |
A Benchmark Dataset for Multi-Level Complexity-Controllable Machine Translation (2022.lrec-1)
Copied to clipboard
Kazuki Tani, Ryoya Yuasa, Kazuki Takikawa, Akihiro Tamura, Tomoyuki Kajiwara, Takashi Ninomiya, Tsuneo Kato
| Challenge: | Existing test datasets for MLCC-MT have three problems: A source language sentence and its simplified target language sentence are not necessarily exactly parallel. |
| Approach: | They propose to use a test dataset to evaluate multi-level complexity-controllable machine translation (MLCC-MT) their results are compared to a standard test dataset constructed from the Newsela corpus . |
| Outcome: | The proposed test dataset is based on the Newsela corpus and is released . it includes automatic filtering, manual check for parallel target language sentences . |
Exploiting Target Language Data for Neural Machine Translation Beyond Back Translation (2024.findings-acl)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) encounters challenges when translating in new domains and low-resource languages. |
| Approach: | They propose a variant of k-nearest neighbor machine translation that utilizes target language data by constructing a pseudo datastore. |
| Outcome: | The proposed method exhibits strong domain adaptation capability in both high-resource and low-resourced machine translation. |
Addressing Posterior Collapse with Mutual Information for Improved Variational Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing variational inference models ignore their latent variables, a phenomenon called posterior collapse. |
| Approach: | They propose a new loss function for conditional variational autoencoders that counteracts posterior collapse by using a modified evidence lower bound objective and a factorized decoder. |
| Outcome: | The proposed model yields improved translation quality compared to existing models on WMT RoEn and DeEn. |
Sampling-Based Approximations to Minimum Bayes Risk Decoding for Neural Machine Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to improve beam search quality are inadequate in many ways . a new approximation to the beam search curse has been proposed . |
| Approach: | They propose an approximation to minimum Bayes risk decoding that would solve the beam search curse. |
| Outcome: | The proposed approximation has no equivalent to the beam search curse. |
Optimal Transport for Unsupervised Hallucination Detection in Neural Machine Translation (2023.acl-long)
Copied to clipboard
| Challenge: | Neural machine translation models can unpredictably produce severely pathological translations, known as hallucinations, that seriously undermine user trust. |
| Approach: | They propose a fully unsupervised, plug-in detector that can be used with any attention-based NMT model. |
| Outcome: | The proposed detector outperforms existing models and is competitive with detectors that employ external models trained on millions of samples. |
Improving Machine Translation with Large Language Models: A Preliminary Study with Cooperative Decoding (2024.findings-acl)
Copied to clipboard
| Challenge: | Commercial machine translation engines are proficient in addressing the majority of translation requirements. |
| Approach: | They propose to combine NMT and MT-oriented LLMs to achieve superior translation quality by combining their strengths. |
| Outcome: | The proposed model can handle complex scenarios beyond the capability of NMT alone. |
Neural Machine Translation Quality and Post-Editing Performance (2021.emnlp-main)
Copied to clipboard
| Challenge: | a recent study has shown that MT post-editing can reduce translation quality and speed . a large-scale study involving 30 professional translators examined the relationship between MT performance and post-edited outputs. |
| Approach: | They examine the relationship between MT performance and post-editing time and quality . they use neural MT of high quality to improve translation quality based on phrase-based MT . |
| Outcome: | The proposed model is not stable predictor of time or quality, the authors say . they find that better MT systems lead to fewer changes in the sentences . |
Leveraging GPT-4 for Automatic Translation Post-Editing (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Neural Machine Translation models still require translation post-editing to rectify errors and enhance quality under critical settings. |
| Approach: | They use GPT-4 to automatically post-edit NMT outputs across several language pairs . they show that GPT4 is adept at translation post- editing, producing meaningful edits . |
| Outcome: | The proposed translation post-editor improves on state-of-the-art language models on English-Chinese, English-German, Chinese-English and German-English language pairs. |
Efficient Domain Adaptation for Non-Autoregressive Machine Translation (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing non-parametric approaches like nearest neighbor machine translation have made small Autoregressive translation models less efficient . despite their impressive generalization and task performance, LLMs suffer from prohibitive inference cost when confronted with specific domains. |
| Approach: | They propose a domain adaptation approach that tailors a k-nearest-neighbor algorithm for NAT models that incorporates the parallel nature of NAT. |
| Outcome: | The proposed approach achieves significant improvements over the Base-NAT model and exhibits enhanced efficiency. |
On Search Strategies for Document-Level Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Document-level neural machine translation models produce a more consistent output across a document . however, the exact decoding strategy is often not described and not mentioned at all. |
| Approach: | They propose to use standard automatic metrics and specific linguistic phenomena to compare different decoding schemes. |
| Outcome: | The proposed decoding strategies perform similar to each other on three standard document-level translation benchmarks. |
Digging Errors in NMT: Evaluating and Understanding Model Errors from Partial Hypothesis Space (2022.emnlp-main)
Copied to clipboard
| Challenge: | Current evaluation of neural machine translation systems is limited by one best hypothesis and search errors brought by heuristic decoding algorithms. |
| Approach: | They propose a new evaluation protocol which defines model errors with model’s ranking capability over hypothesis space and Monte Carlo sampling evaluation to tackle the problem of exponentially large space. |
| Outcome: | The proposed evaluation protocol is consistent with what is currently used in the field and is consistent to what is being proposed. |
Towards a Better Understanding of Variations in Zero-Shot Neural Machine Translation Performance (2023.emnlp-main)
Copied to clipboard
| Challenge: | Prior work has investigated causes of poor zero-shot performance, but new study suggests it does not exhibit poor zero shot capability. |
| Approach: | They propose to investigate the presence of significant variations in zero-shot performance . target-side translation quality is most influential factor, with vocabulary overlap impacting zero- shot capabilities . |
| Outcome: | The results show that the target side translation quality is the most influential factor . linguistic properties, such as language family and writing system, play a role . |
Code-Switching with Word Senses for Pretraining in Neural Machine Translation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Lexical ambiguity is a significant and pervasive challenge in Neural Machine Translation (NMT) many state-of-the-art (SOTA) NMT systems struggle to handle polysemous words . |
| Approach: | They propose an end-to-end approach for pretraining multilingual NMT models leveraging word sense-specific information from Knowledge Bases. |
| Outcome: | The proposed approach improves translation quality and scales to various data and resource-strapped scenarios. |
Finding the Optimal Byte-Pair Encoding Merge Operations for Neural Machine Translation in a Low-Resource Setting (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Using different byte pair encoder configurations, we can improve neural machine translation performance for low-resource languages. |
| Approach: | They investigate the impact of different Byte Pair Encoding configurations on neural machine translation performance for the Filipino-Cebuano language pair across various text domains. |
| Outcome: | The proposed methods show that smaller BPE configurations yield higher BLEU scores, indicating improved translation quality through finer tokenization granularity . larger BPE setups and the absence of BPE result in lower BLUE scores, suggesting a decline in translation quality due to coarser tokenisation. |
Nearest Neighbor Machine Translation is Meta-Optimizer on Output Projection Layer (2023.emnlp-main)
Copied to clipboard
| Challenge: | Nearest Neighbor Machine Translation (kNN-MT) is a powerful domain adaptation tool . the reasons for its success have not been thoroughly investigated . |
| Approach: | They propose to integrate pre-trained Neural Machine Translation models with token-level retrieval . they propose to implicitly execute gradient descent on the output projection layer of NMT . |
| Outcome: | The proposed approach outperforms model fine-tuning on in-domain tests while achieving better performance on out-of-domain sets. |
In-Image Neural Machine Translation with Segmented Pixel Sequence-to-Sequence Model (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In-Image Machine Translation (IIMT) aims to convert images containing texts from one language to another. |
| Approach: | They propose an end-to-end model instead of the traditional cascade methods which use optical character recognition followed by neural machine translation and text rendering. |
| Outcome: | The proposed model outperforms both cascade methods and current model in translation quality and robustness across various dimensions. |
MoNMT: Modularly Leveraging Monolingual and Bilingual Knowledge for Neural Machine Translation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing models for multi-domain translation tasks only use monolingual data, whereas bilingual data is indispensable for improving the models. |
| Approach: | They propose a modular strategy that facilitates the cooperation of monolingual and bilingual knowledge in translation tasks by avoiding catastrophic forgetting. |
| Outcome: | The proposed model exhibits superior generalization and robustness over the conventional approach. |
THOR-MoE: Hierarchical Task-Guided and Context-Responsive Routing for Neural Machine Translation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing sparse Mixture-of-Experts (MoE) solutions may lead to sub-optimal performance . thor-moe uses domain/linguistics-specific knowledge, but lacks context-responsive routing policies . |
| Approach: | They propose a sparse Mixture-of-Experts (MoE) solution which uses task knowledge of NMT into MoE and provides hierarchical task-guided and context-responsive routing policies. |
| Outcome: | thor-MoE can achieve an average improvement of 0.75 BLEU with less than 22% activated parameters on multi-domain translation tasks. |
Neural Machine Translation between Low-Resource Languages with Synthetic Pivoting (2024.lrec-main)
Copied to clipboard
| Challenge: | Pivot-based neural machine translation systems overcome data scarcity by including a high-resource pivot language in the process of translating between low-resourced languages. |
| Approach: | They propose a novel approach to pivot-based translation in which pivot sentences are generated synthetically from both the source and target languages. |
| Outcome: | The proposed approach improves pivot-based systems translating between low-resource Southern African languages by up to 5.6 BLEU points after fine-tuning. |
Multi-perspective Alignment for Increasing Naturalness in Neural Machine Translation (2025.acl-long)
Copied to clipboard
| Challenge: | Neural machine translation systems amplify lexical biases, rendering outputs artificially impoverished . Attempts to increase naturalness in NMT can fall short in terms of content preservation . |
| Approach: | They propose a method that rewards both naturalness and content preservation . they use multiple perspectives to produce more natural translations . |
| Outcome: | The proposed method produces translations that are lexically richer and exhibit more properties of human-written language without loss in translation accuracy. |
AFRIDOC-MT: Document-level MT Corpus for African Languages (2025.emnlp-main)
Copied to clipboard
Jesujoba Oluwadara Alabi, Israel Abebe Azime, Miaoran Zhang, Cristina España-Bonet, Rachel Bawden, Dawei Zhu, David Ifeoluwa Adelani, Clement Oyeleke Odoje, Idris Akinade, Iffat Maab, Davis David, Shamsuddeen Hassan Muhammad, Neo Putini, David O. Ademuyiwa, Andrew Caines, Dietrich Klakow
| Challenge: | AFRIDOC-MT is a document-level multi-parallel translation dataset covering five languages . AFRITIC-MT models perform better on sentences than general-purpose LLMs . |
| Approach: | They propose a document-level multi-parallel translation dataset covering English and five African languages. |
| Outcome: | The proposed dataset covers 334 health and 271 information technology news documents . it shows that NLLB-200 achieves the best average performance among standard models . |