Breaking the Beam Search Curse: A Study of (Re-)Scoring Methods and Stopping Criteria for Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Beam search is widely used in neural machine translation, but beam sizes larger than 5 hurt translation quality. |
| Approach: | They propose to use beam search to improve translation quality by using hyperparameter-free methods that outperform the widely-used heuristic of length normalization by +2.0 BLEU. |
| Outcome: | The proposed methods outperform the widely-used heuristic on Chinese-to-English translation and achieve the best results among all methods. |
Similar Papers
Learning to Stop in Structured Prediction for Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | Beam search optimization solves many problems in neural machine translation, but lacks principled stopping criteria and does not learn how to stop during training. |
| Approach: | They propose a ranking method which enables an optimal beam search stop-ping criteria and a structured prediction loss function which penalizes suboptimal finished candidates produced by beam search during training. |
| Outcome: | Experiments on synthetic and real languages show that the proposed methods improve translation quality and length. |
On NMT Search Errors and Model Errors: Cat Got Your Tongue? (D19-1)
Copied to clipboard
| Challenge: | We show that at the root of the problem of empty translations lies an inherent bias towards shorter translations. |
| Approach: | They propose an exact inference procedure for neural sequence models based on beam search and depth-first search. |
| Outcome: | The proposed procedure finds that beam search fails to find the best model scores . the results show that the model often prefers an empty translation . |
Improving Beam Search by Removing Monotonic Constraint for Neural Machine Translation (P18-2)
Copied to clipboard
| Challenge: | a beam search algorithm produces monotonic left-to-right order, meaning a hypothesis cannot be revisited . a proposed algorithm allows discarded hypotheses to be recovered in a later step. |
| Approach: | They propose to decode a beam search algorithm that considers multiple hypotheses simultaneously . they propose to maintain all found hypothese a single priority queue and a universal score function . |
| Outcome: | The proposed algorithm improves translations even for high-performance models in English-Japanese translation task. |
First the Worst: Finding Better Gender Translations During Beam Search (2022.findings-acl)
Copied to clipboard
| Challenge: | Neural language generation models optimized by likelihood tend towards 'safe' word choice. |
| Approach: | They propose to use beam search to improve gender diversity in n-best lists and rerank n best lists using gender features obtained from the source sentence to address this problem. |
| Outcome: | The proposed approach improves gender diversity in n-best lists and reranks n best lists using gender features obtained from the source sentence. |
Machine Translation Decoding beyond Beam Search (2021.emnlp-main)
Copied to clipboard
Rémi Leblond, Jean-Baptiste Alayrac, Laurent Sifre, Miruna Pislar, Lespiau Jean-Baptiste, Ioannis Antonoglou, Karen Simonyan, Oriol Vinyals
| Challenge: | a new study examines whether beam search can be replaced by a more powerful metric-driven search technique. |
| Approach: | They propose a beam search method which is agnostic to the end metric and report results on a variety of metrics. |
| Outcome: | The proposed method is based on a Monte-Carlo Tree Search (MCTS) based method and shows it can be used in language applications. |
If beam search is the answer, what was the question? (2020.emnlp-main)
Copied to clipboard
| Challenge: | surprisingly, beam search results on language generation tasks are low-quality . despite its high error rate, beam searches can be used to decode models with high probability . |
| Approach: | They frame beam search as the exact solution to a different decoding objective . they propose a set of decoding objectives that explicitly enforce this property . |
| Outcome: | The proposed method enforces uniform information density in text, a property motivated by cognitive science. |
A Simple and Effective Approach to Coverage-Aware Neural Machine Translation (P18-2)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) models are used to solve translation problems using long-term models. |
| Approach: | They propose a method to seek a better balance between model confidence and length preference for Neural Machine Translation. |
| Outcome: | The proposed model improves on Chinese-English and English-German translation tasks. |
Multi-Sentence Resampling: A Simple Approach to Alleviate Dataset Length Bias and Beam-Search Degradation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Neural Machine Translation suffers from a beam-search problem after a certain point, especially for long sentences. |
| Approach: | They propose a data augmentation technique that concatenates several sentences from the original dataset to make a long training example. |
| Outcome: | The proposed technique significantly reduces degradation with growing beam size and improves translation quality on the IWSTL15 En-Vi, IWStl17 En-Fr, and WMT14 En-De datasets. |
Incremental Beam Manipulation for Natural Language Generation (2021.eacl-main)
Copied to clipboard
| Challenge: | a larger beam size can lead to deteriorating performance of natural language generation systems due to model errors . performance of NLG systems can plateau or even decrease when beam sizes larger than 10 are used . |
| Approach: | They propose to rerank the output of beam search to produce a good set of hypotheses . they propose incremental beam manipulation to discarded hypothese . |
| Outcome: | The proposed method outperforms a strong reranker on the E2E and WebNLG datasets while being on par with the existing method. |
Sampling-Based Approximations to Minimum Bayes Risk Decoding for Neural Machine Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to improve beam search quality are inadequate in many ways . a new approximation to the beam search curse has been proposed . |
| Approach: | They propose an approximation to minimum Bayes risk decoding that would solve the beam search curse. |
| Outcome: | The proposed approximation has no equivalent to the beam search curse. |