Improving Beam Search by Removing Monotonic Constraint for Neural Machine Translation (P18-2)
Copied to clipboard
| Challenge: | a beam search algorithm produces monotonic left-to-right order, meaning a hypothesis cannot be revisited . a proposed algorithm allows discarded hypotheses to be recovered in a later step. |
| Approach: | They propose to decode a beam search algorithm that considers multiple hypotheses simultaneously . they propose to maintain all found hypothese a single priority queue and a universal score function . |
| Outcome: | The proposed algorithm improves translations even for high-performance models in English-Japanese translation task. |
Similar Papers
Breaking the Beam Search Curse: A Study of (Re-)Scoring Methods and Stopping Criteria for Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Beam search is widely used in neural machine translation, but beam sizes larger than 5 hurt translation quality. |
| Approach: | They propose to use beam search to improve translation quality by using hyperparameter-free methods that outperform the widely-used heuristic of length normalization by +2.0 BLEU. |
| Outcome: | The proposed methods outperform the widely-used heuristic on Chinese-to-English translation and achieve the best results among all methods. |
Machine Translation Decoding beyond Beam Search (2021.emnlp-main)
Copied to clipboard
Rémi Leblond, Jean-Baptiste Alayrac, Laurent Sifre, Miruna Pislar, Lespiau Jean-Baptiste, Ioannis Antonoglou, Karen Simonyan, Oriol Vinyals
| Challenge: | a new study examines whether beam search can be replaced by a more powerful metric-driven search technique. |
| Approach: | They propose a beam search method which is agnostic to the end metric and report results on a variety of metrics. |
| Outcome: | The proposed method is based on a Monte-Carlo Tree Search (MCTS) based method and shows it can be used in language applications. |
Learning to Stop in Structured Prediction for Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | Beam search optimization solves many problems in neural machine translation, but lacks principled stopping criteria and does not learn how to stop during training. |
| Approach: | They propose a ranking method which enables an optimal beam search stop-ping criteria and a structured prediction loss function which penalizes suboptimal finished candidates produced by beam search during training. |
| Outcome: | Experiments on synthetic and real languages show that the proposed methods improve translation quality and length. |
Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation (N18-1)
Copied to clipboard
| Challenge: | Existing approaches to neural machine translation have computational complexities that are either linear or exponential in the number of constraints. |
| Approach: | They propose an algorithm for lexically constrained decoding with a complexity of O(1) in the number of constraints. |
| Outcome: | The proposed algorithm can place constraints and improve results in simulated post-editing tasks. |
Exploring Recombination for Efficient Decoding of Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) decoder captures features of entire prediction history . some partial hypotheses with different prefixes will be regarded differently no matter how similar they are . |
| Approach: | They propose a method that uses a n-gram suffix to adapt it to beam search decoding. |
| Outcome: | The proposed method can obtain similar translation quality with a smaller beam size, making it more efficient. |
Restricted or Not: A General Training Framework for Neural Machine Translation (2022.acl-srw)
Copied to clipboard
| Challenge: | Existing work imposes constraints on beam search decoding, which limits the concurrent processing ability of the model in deployment. |
| Approach: | They propose a general training framework that allows a model to support both restricted and unrestricted translations by adopting an additional auxiliary training process without constraining the decoding process. |
| Outcome: | The proposed training framework is tested on simulated and original benchmarks. |
If beam search is the answer, what was the question? (2020.emnlp-main)
Copied to clipboard
| Challenge: | surprisingly, beam search results on language generation tasks are low-quality . despite its high error rate, beam searches can be used to decode models with high probability . |
| Approach: | They frame beam search as the exact solution to a different decoding objective . they propose a set of decoding objectives that explicitly enforce this property . |
| Outcome: | The proposed method enforces uniform information density in text, a property motivated by cognitive science. |
A Call for Clarity in Beam Search: How It Works and When It Stops (2024.lrec-main)
Copied to clipboard
| Challenge: | Empirical results show that a modified beam decoding implementation improves decoding performance of strong, neural language generation models. |
| Approach: | They propose a modification to a beam decoding implementation that generalizes the stopping criterion and provides flexibility to the depth of search. |
| Outcome: | The proposed method improves decoding performance of strong models on news text summarization and machine translation over diverse language pairs with negligible inference slowdown. |
First the Worst: Finding Better Gender Translations During Beam Search (2022.findings-acl)
Copied to clipboard
| Challenge: | Neural language generation models optimized by likelihood tend towards 'safe' word choice. |
| Approach: | They propose to use beam search to improve gender diversity in n-best lists and rerank n best lists using gender features obtained from the source sentence to address this problem. |
| Outcome: | The proposed approach improves gender diversity in n-best lists and reranks n best lists using gender features obtained from the source sentence. |
Best-First Beam Search (2020.tacl-1)
Copied to clipboard
| Challenge: | Currently, beam search is the default for decoding structured predictors . however, little work has been done to speed up beam search itself . |
| Approach: | They propose a beam search algorithm that prunes the scoring function to a monotonic sequence length, which allows for safe pruning of hypotheses that cannot be in the final set of hypothecies. |
| Outcome: | The proposed method can be implemented up to 10x faster in practice. |