Accelerating NMT Batched Beam Decoding with LMBR Posteriors for Deployment (N18-3)
Copied to clipboard
| Challenge: | LMBR techniques for NMT still yield better results than Transformers . but with NMT, real time decoding is challenging without GPUs and high-end GPUs are expensive. |
| Approach: | They propose a batched beam decoding algorithm for NMT with LMBR n-gram posteriors and an acceleration strategy for deployment to take advantage of the higher adequacy. |
| Outcome: | The proposed method outperforms the most recent results with Transformers in terms of speed and memory usage. |
Similar Papers
A Streaming Approach For Efficient Batched Beam Search (2020.emnlp-main)
Copied to clipboard
| Challenge: | During decoding, candidates terminate or are pruned according to heuristics, a streaming method is used to "refill" the batch after it finishes translating some fraction of the current batch. |
| Approach: | They propose an efficient batching strategy for variable-length decoding on GPU architectures by streamlining the batching process. |
| Outcome: | The proposed method reduces runtime by 71% compared to a fixed-width beam search baseline and 17% compared with a variable-widness baseline while matching baselines’ BLEU. |
Exploring Recombination for Efficient Decoding of Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) decoder captures features of entire prediction history . some partial hypotheses with different prefixes will be regarded differently no matter how similar they are . |
| Approach: | They propose a method that uses a n-gram suffix to adapt it to beam search decoding. |
| Outcome: | The proposed method can obtain similar translation quality with a smaller beam size, making it more efficient. |
Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation (N18-1)
Copied to clipboard
| Challenge: | Existing approaches to neural machine translation have computational complexities that are either linear or exponential in the number of constraints. |
| Approach: | They propose an algorithm for lexically constrained decoding with a complexity of O(1) in the number of constraints. |
| Outcome: | The proposed algorithm can place constraints and improve results in simulated post-editing tasks. |
Quality-Aware Decoding for Neural Machine Translation (2022.naacl-main)
Copied to clipboard
Patrick Fernandes, António Farinhas, Ricardo Rei, José G. C. de Souza, Perez Ogayo, Graham Neubig, Andre Martins
| Challenge: | Despite advances in machine translation quality estimation and evaluation, decoding is mostly oblivious to this. |
| Approach: | They propose to use a decoding framework that is quality-aware for neural machine translation . they compare various methods like N-best reranking and minimum Bayes risk decoding . |
| Outcome: | The proposed quality-aware decoding outperforms MAP-based decoding on four datasets and two model classes. |
Language-Informed Beam Search Decoding for Multilingual Machine Translation (2024.findings-acl)
Copied to clipboard
| Challenge: | Beam search decoding is the de-facto method for decoding auto-regressive Neural Machine Translation (NMT) models, but decoding multilingual NMT models produces off-target translations . |
| Approach: | They propose a general decoding algorithm incorporating an off-the-shelf Language Identification (LiD) model into beam search decoding to reduce off-target translations. |
| Outcome: | The proposed language-informed beam search improves +1.1 BLEU and +0.9 BLUE on WMT and OPUS datasets and reduces off-target rates from 22.9% to 7.7% and 65.8% to 25.3% respectively. |
Dynamic Oracle for Neural Machine Translation in Decoding Phase (L18-1)
Copied to clipboard
| Challenge: | Existing methods to improve NMT performance but there is a discrepancy between training and inference when decoding. |
| Approach: | They propose to use Scheduled Sampling to reduce the discrepancy between training and inference in NMT when decoding to mitigate the discrépancy. |
| Outcome: | The proposed methods improve translation quality over standard NMT system. |
MobileNMT: Enabling Translation in 15MB and 30ms (2023.acl-industry)
Copied to clipboard
| Challenge: | Existing work on NMT models is limited in storage, memory, computation and power consumption. |
| Approach: | They propose a mobile machine translation system that can translate in 15MB and 30ms on devices. |
| Outcome: | The proposed system can translate in 15MB and 30ms on mobile devices. |
Compressing Large-Scale Transformer-Based Models: A Case Study on BERT (2021.tacl-1)
Copied to clipboard
Prakhar Ganesh, Yao Chen, Xin Lou, Mohammad Ali Khan, Yin Yang, Hassan Sajjad, Preslav Nakov, Deming Chen, Marianne Winslett
| Challenge: | Popular pre-trained Transformers have improved performance for various NLP tasks by sizable margins, but are too resource-hungry and computation-intensive to suit low-capacity devices or applications with strict latency requirements. |
| Approach: | They present a literature review of the compression of Transformers, focusing on the popular BERT model, which has attracted considerable research attention. |
| Outcome: | The proposed models improve Sentiment analysis, paraphrase detection, machine reading comprehension, question answering, text summarization, and other tasks by sizable margins. |
Efficient Beam Search for Large Language Models Using Trie-Based Decoding (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) face memorybound performance bottlenecks due to their high memory requirements. |
| Approach: | They propose a trie-based parallel decoding method that shares a single KV cache across beams with common prefixes to dramatically reduce memory usage and enables efficient decoding. |
| Outcome: | The proposed method significantly reduces memory usage and enables efficient decoding without compromising generation quality. |
Multi-Sentence Resampling: A Simple Approach to Alleviate Dataset Length Bias and Beam-Search Degradation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Neural Machine Translation suffers from a beam-search problem after a certain point, especially for long sentences. |
| Approach: | They propose a data augmentation technique that concatenates several sentences from the original dataset to make a long training example. |
| Outcome: | The proposed technique significantly reduces degradation with growing beam size and improves translation quality on the IWSTL15 En-Vi, IWStl17 En-Fr, and WMT14 En-De datasets. |