| Challenge: | Existing methods to generate multiple translation candidates do not address the overcorrection problem, which discourages the model from generating synonymous expressions and leans toward gold standards, reducing the diversity in the candidates. |
| Approach: | They propose to introduce perturbed k-nearest neighbor machine translation (kNN-MT) to generate more diverse translations. |
| Outcome: | The proposed methods significantly improve candidate diversity and control diversity by tuning the perturbation’s magnitude. |
Similar Papers
Twist Decoding: Diverse Generators Guide Each Other (2022.emnlp-main)
Copied to clipboard
Jungo Kasai, Keisuke Sakaguchi, Ronan Le Bras, Hao Peng, Ximing Lu, Dragomir Radev, Yejin Choi, Noah A. Smith
| Challenge: | Using a variety of language generation models, ensembling models is challenging during inference. |
| Approach: | They propose a method that decodes text models that do not assume a shared vocabulary, tokenization or generation order. |
| Outcome: | The proposed method outperforms models decoded in isolation over various scenarios. |
Adaptive Nearest Neighbor Machine Translation (2021.acl-short)
Copied to clipboard
| Challenge: | kNN-MT uses pre-trained NMT model with token-level k-nearest-neighbor retrieval to improve translation accuracy. |
| Approach: | They propose a method that combines a pre-trained NMT model with token-level k-nearest-neighbor retrieval to improve translation accuracy. |
| Outcome: | The proposed method outperforms the existing model on four benchmark datasets and is open-source. |
Towards Robust k-Nearest-Neighbor Machine Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | k-Nearest-Neighbor Machine Translation (kNN-MT) is a popular research paradigm in machine translation. |
| Approach: | They propose a confidence-enhanced kNN-MT model with robust training to reduce noise . they introduce NMT confidence to refine the modeling of important components of kN-MT . |
| Outcome: | The proposed model improves on four benchmark datasets and is robust to training. |
Mixup Decoding for Diverse Machine Translation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for generating multiple translations for source and target languages neglect the one-to-many mapping between the source and the target languages. |
| Approach: | They propose a method to generate different translations for the input sentence by linearly interpolating it with different sentence pairs sampled from the training corpus during decoding. |
| Outcome: | Experiments on WMT’16 en-ro, WMT'14 en de, and WMT ‘17 zh-en show that the proposed method outperforms all previous diverse machine translation methods. |
Nearest Neighbor Machine Translation is Meta-Optimizer on Output Projection Layer (2023.emnlp-main)
Copied to clipboard
| Challenge: | Nearest Neighbor Machine Translation (kNN-MT) is a powerful domain adaptation tool . the reasons for its success have not been thoroughly investigated . |
| Approach: | They propose to integrate pre-trained Neural Machine Translation models with token-level retrieval . they propose to implicitly execute gradient descent on the output projection layer of NMT . |
| Outcome: | The proposed approach outperforms model fine-tuning on in-domain tests while achieving better performance on out-of-domain sets. |
kNN-BOX: A Unified Framework for Nearest Neighbor Generation (2024.eacl-demo)
Copied to clipboard
| Challenge: | kNN-BOX enables quick development and visualization for novel generation paradigm . Currently, knn-BOx has provided implementation of seven popular kN-MT variants . |
| Approach: | They propose a framework which decomposes the datastore-augmentation approach into three modules . they apply kNN-BOX to machine translation and three other tasks . |
| Outcome: | The proposed framework decomposes the datastore-augmentation approach into three modules . it provides implementation of seven popular kNN-MT variants, covering research from performance enhancement to efficiency optimization. |
Revisiting Source Context in Nearest Neighbor Machine Translation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing research does not explicitly consider the source context when retrieving similar examples . |
| Approach: | They propose a method to improve neural machine translation via source context enhancement by integrating a source-aware distance calibration module. |
| Outcome: | The proposed approach can be integrated with representative kNN-MT baselines and achieve significant performance improvements. |
Machine Translation into Low-resource Language Varieties (2021.acl-short)
Copied to clipboard
| Challenge: | Current machine translation systems generate a "standard" target language, but many languages have multiple varieties that are different from the standard language. |
| Approach: | They propose a framework to rapidly adapt machine translation systems to generate different target varieties . they propose to use no parallel data to generate languages close to, but different from, the standard target language . |
| Outcome: | The proposed model improves on a system that generates Ukrainian and Belarusian in two languages with no parallel data. |
Breeding Machine Translations: Evolutionary approach to survive and thrive in the world of automated evaluation (2023.acl-long)
Copied to clipboard
| Challenge: | a genetic algorithm (GA) based method improves MT quality and identifies weaknesses in evaluation metrics. |
| Approach: | They propose a genetic algorithm-based method for modifying n-best lists produced by a machine translation system using a fitness function. |
| Outcome: | The proposed method improves translation quality and identifies weaknesses in evaluation metrics. |
Comparison of Diverse Decoding Methods from Conditional Language Models (P19-1)
Copied to clipboard
| Challenge: | Conditional language models can generate a diverse set of outputs, but for open-ended tasks, beam search is ill-suited to generating a set of diverse sequences. |
| Approach: | They propose a method where we over-sample candidates and use clustering to remove similar sequences to achieve high diversity without sacrificing quality. |
| Outcome: | The proposed method over-samples candidates and removes similar sequences to achieve high diversity without sacrificing quality. |