Bilingual Mutual Information Based Adaptive Training for Neural Machine Translation (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to token-level adaptive training only use static word frequency information without considering the source language. |
| Approach: | They propose a bilingual mutual information based adaptive objective that assigns weights to target tokens with higher BMI . they propose to use this approach to improve token-level adaptive training . |
| Outcome: | The proposed method improves token-level adaptive training on two languages. |
Similar Papers
Conditional Bilingual Mutual Information Based Adaptive Training for Neural Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to improve neural machine translation use token-level adaptive training . however, standard models make predictions on condition of previous contexts . |
| Approach: | They propose a target-context-aware metric which can be supplemented by statistical metrics . they propose an adaptive training approach based on token- and sentence-level CBMI . |
| Outcome: | The proposed model outperforms the Transformer baseline and other similar approaches on English-German and Chinese-English tasks. |
Normalizing Mutual Information for Robust Adaptive Training for Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Neural machine translation models have been reported to generate hallucinations . despite the success of the models, there are still challenges to improve fluency . |
| Approach: | They propose a scoring metric for the importance of target sentences and tokens to encourage fluent translations. |
| Outcome: | The proposed metric improves translation fluency and source-faithfulness . the proposed nmi model is not properly normalized, the authors argue . |
Token-level Adaptive Training for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing token imbalance phenomenon in natural language as different tokens appear with different frequencies, which leads to different learning difficulties for tokens in Neural machine translation (NMT). |
| Approach: | They propose to assign tokens with different frequencies to target tokens during training to encourage the model to pay more attention to low-frequency tokens. |
| Outcome: | The proposed model yields consistent improvements on ZH-EN, EN-RO, and EN-DE translation tasks, especially on sentences that contain more low-frequency tokens. |
Multi-Agent Mutual Learning at Sentence-Level and Token-Level for Neural Machine Translation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) has achieved significant progress over recent years. |
| Approach: | They extend mutual learning to the machine translation task and operate at both the sentence-level and the token-level. |
| Outcome: | The proposed method improves on the IWSLT’14 German-English task and also on the WMT’14 English-German task. |
Token-Level Self-Evolution Training for Sequence-to-Sequence Learning (2023.acl-short)
Copied to clipboard
| Challenge: | Adaptive training approaches do not consider the variation of learning difficulty in different training steps, making the learning deterministic and sub-optimal. |
| Approach: | They propose a dynamic token-level self-evolution training method that reweighs the training losses of different target tokens based on priors. |
| Outcome: | Empirically, the proposed method yields significant improvements on three translation tasks. |
Adaptive Token-level Cross-lingual Feature Mixing for Multilingual Neural Machine Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual neural machine translation models can translate multiple language pairs in a single model but lacks ability to capture language-specific features. |
| Approach: | They propose a token-level feature mixing method that captures different features and dynamically determines feature sharing across languages. |
| Outcome: | The proposed method outperforms baselines and can be extended to zero-shot translation. |
Self-Vocabularizing Training for Neural Machine Translation (2025.naacl-srw)
Copied to clipboard
| Challenge: | Past vocabulary learning techniques identify relevant vocabulary before training, relying on corpus statistics or frequency counts without considering contextual information or the model's ability to represent it. |
| Approach: | They propose a method that self-vocabularizes a smaller, more optimal vocabulary by pairing source sentences with the model's predictions to define a new vocabulary. |
| Outcome: | The proposed method produces a 1.49 BLEU improvement in the simulated model and an increase in unique token usage and a 6–8% reduction in vocabulary size. |
It’s Easier to Translate out of English than into it: Measuring Neural Translation Difficulty by Cross-Mutual Information (2020.acl-main)
Copied to clipboard
| Challenge: | Current state-of-the-art MT systems are based on neural networks, but it is unclear whether all translation directions are equally easy (or hard) to model for NMT. |
| Approach: | They propose an asymmetric information-theoretic metric of machine translation difficulty that exploits the probabilistic nature of most neural machine translation models. |
| Outcome: | The proposed metric allows us to better evaluate the difficulty of translating text into the target language while controlling for the difficulty independent of the translation task. |
Integrating Multi-scale Contextualized Information for Byte-based Neural Machine Translation (2024.findings-acl)
Copied to clipboard
| Challenge: | Subword tokenization is a common method for vocabulary building in NMT systems . but, it has its disadvantages, such as a vocabulary cannot be modified once it is learned . |
| Approach: | They propose a method that learns contextualized information of varying scales . they propose byte-based tokenization to solve these problems with few embedding parameters . |
| Outcome: | Experiments show that the proposed method outperforms subword-based methods in multilingual and out-of-domain scenarios. |
How Lexical is Bilingual Lexicon Induction? (2024.findings-naacl)
Copied to clipboard
| Challenge: | lexical variation and low-resource settings make it difficult to learn in low-level settings. |
| Approach: | They propose to incorporate additional lexical information into the retrieve-and-rank approach to improve lexicon induction. |
| Outcome: | The proposed approach improves on XLING by an average of 2% across all language pairs. |