| Challenge: | kNN-MT uses pre-trained NMT model with token-level k-nearest-neighbor retrieval to improve translation accuracy. |
| Approach: | They propose a method that combines a pre-trained NMT model with token-level k-nearest-neighbor retrieval to improve translation accuracy. |
| Outcome: | The proposed method outperforms the existing model on four benchmark datasets and is open-source. |
Similar Papers
Efficient k-Nearest-Neighbor Machine Translation with Dynamic Retrieval (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing models for non-parametric domain adaptation lack kNN retrieval at each timestep, leading to substantial time overhead. |
| Approach: | They propose a kNN-MT-based model that uses a domain-specific translation knowledge store to interpolate the prediction distribution of the model. |
| Outcome: | The proposed model significantly extends kNN-MT with dynamic retrieval on widely-used datasets. |
Towards Robust k-Nearest-Neighbor Machine Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | k-Nearest-Neighbor Machine Translation (kNN-MT) is a popular research paradigm in machine translation. |
| Approach: | They propose a confidence-enhanced kNN-MT model with robust training to reduce noise . they introduce NMT confidence to refine the modeling of important components of kN-MT . |
| Outcome: | The proposed model improves on four benchmark datasets and is robust to training. |
Efficient Cluster-Based k-Nearest-Neighbor Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | k-Nearest-Neighbor Machine Translation (kNN-MT) is a non-parametric solution for domain adaptation . previous studies have shown that kNN retrieval is at the expense of high latency . |
| Approach: | They propose to use clustering to improve retrieval efficiency by combining a non-parametric MT with an in-domain feature-based retrieval module. |
| Outcome: | The proposed method reduces translation latency by 57% while maintaining the most useful information of the original datastore. |
Nearest Neighbor Machine Translation is Meta-Optimizer on Output Projection Layer (2023.emnlp-main)
Copied to clipboard
| Challenge: | Nearest Neighbor Machine Translation (kNN-MT) is a powerful domain adaptation tool . the reasons for its success have not been thoroughly investigated . |
| Approach: | They propose to integrate pre-trained Neural Machine Translation models with token-level retrieval . they propose to implicitly execute gradient descent on the output projection layer of NMT . |
| Outcome: | The proposed approach outperforms model fine-tuning on in-domain tests while achieving better performance on out-of-domain sets. |
Subset Retrieval Nearest Neighbor Machine Translation (2023.acl-long)
Copied to clipboard
| Challenge: | k-nearest-neighbor machine translation (kNN-MT) is a new approach to improve NMT performance without additional training. |
| Approach: | They propose a method that integrates example-search into the decoding algorithm to improve neighbor token retrieval. |
| Outcome: | The proposed method achieves a speed-up of up to 132.2 times and an improvement in BLEU score of up 1.6 compared with kNN-MT in the WMT’19 translation task and the domain adaptation tasks in De-En and En-Ja. |
Fast Nearest Neighbor Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Fast kNN-MT uses the entire corpus as the datastore for the nearest neighbor search . knn-MT is two-orders slower than vanilla MT models . |
| Approach: | They propose a fast kNN-MT model that uses the entire corpus as the datastore for nearest neighbor search. |
| Outcome: | The proposed model is two-orders faster than kNN-MT and is only two times slower than the standard model. |
Chunk-based Nearest Neighbor Machine Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Semi-parametric models augment generation with retrieval, but require expensive retrieval operation for every generated token. |
| Approach: | They propose a semi-parametric model which augments generation with retrieval by retrieving tokens from a datastore. |
| Outcome: | The proposed model can retrieve chunks of tokens from the datastore, instead of a single token, with a low decoding speed. |
Revisiting Source Context in Nearest Neighbor Machine Translation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing research does not explicitly consider the source context when retrieving similar examples . |
| Approach: | They propose a method to improve neural machine translation via source context enhancement by integrating a source-aware distance calibration module. |
| Outcome: | The proposed approach can be integrated with representative kNN-MT baselines and achieve significant performance improvements. |
Exploiting Target Language Data for Neural Machine Translation Beyond Back Translation (2024.findings-acl)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) encounters challenges when translating in new domains and low-resource languages. |
| Approach: | They propose a variant of k-nearest neighbor machine translation that utilizes target language data by constructing a pseudo datastore. |
| Outcome: | The proposed method exhibits strong domain adaptation capability in both high-resource and low-resourced machine translation. |
Nearest Neighbor Knowledge Distillation for Neural Machine Translation (2022.naacl-main)
Copied to clipboard
| Challenge: | k-nearest-neighbor machine translation (kNN-MT) is a state-of-the-art machine translation technique . however, it requires conducting kNN searches for each decoding step, which increases the cost of decoding . |
| Approach: | They propose to move the time-consuming kNN search forward to the preprocessing phase and introduce k Nearest Neighbor Knowledge Distillation (kNN-KD) that trains the base NMT model to directly learn the knowledge of kN. |
| Outcome: | The proposed method improves over the state-of-the-art model while maintaining the same training and decoding speed as the standard model. |