Variable beam search for generative neural parsing and its relevance for the analysis of neuro-imaging signal (D19-1)
Copied to clipboard
| Challenge: | a variable beam size inference method is proposed for generative parsing for RNNG . the proposed method is not sensitive to lexical biases faced by standard beam search . |
| Approach: | They propose a method of variable beam size inference for Recurrent Neural Network Grammar by drawing inspiration from sequential Monte-Carlo methods such as particle filtering. |
| Outcome: | The proposed method is based on a generative parsing framework that can be used to model brain activity during online sentence comprehension. |
Similar Papers
Finding syntax in human encephalography with beam search (P18-1)
Copied to clipboard
| Challenge: | RNNGs are generative models of (tree , string ) pairs that evaluate derivational choices . a non-syntactic neural language model yields no reliable effects . |
| Approach: | They propose to combine a probabilistic generative grammar with a parsing procedure that uses it to manage syntactic derivations as it advances from one word to the next. |
| Outcome: | The proposed model derives two amplitude effects when used against human encephalography data. |
Effective Batching for Recurrent Neural Network Grammars (2021.findings-acl)
Copied to clipboard
| Challenge: | RNNGs are hard to scale due to the difficulty of batched training. |
| Approach: | They propose to batch RNNGs where every operation is computed in parallel with tensors across multiple sentences. |
| Outcome: | The proposed RNNG scales faster than existing models and achieves x6 speedup compared to existing C++ DyNet implementation . |
Breaking the Beam Search Curse: A Study of (Re-)Scoring Methods and Stopping Criteria for Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Beam search is widely used in neural machine translation, but beam sizes larger than 5 hurt translation quality. |
| Approach: | They propose to use beam search to improve translation quality by using hyperparameter-free methods that outperform the widely-used heuristic of length normalization by +2.0 BLEU. |
| Outcome: | The proposed methods outperform the widely-used heuristic on Chinese-to-English translation and achieve the best results among all methods. |
If beam search is the answer, what was the question? (2020.emnlp-main)
Copied to clipboard
| Challenge: | surprisingly, beam search results on language generation tasks are low-quality . despite its high error rate, beam searches can be used to decode models with high probability . |
| Approach: | They frame beam search as the exact solution to a different decoding objective . they propose a set of decoding objectives that explicitly enforce this property . |
| Outcome: | The proposed method enforces uniform information density in text, a property motivated by cognitive science. |
Machine Translation Decoding beyond Beam Search (2021.emnlp-main)
Copied to clipboard
Rémi Leblond, Jean-Baptiste Alayrac, Laurent Sifre, Miruna Pislar, Lespiau Jean-Baptiste, Ioannis Antonoglou, Karen Simonyan, Oriol Vinyals
| Challenge: | a new study examines whether beam search can be replaced by a more powerful metric-driven search technique. |
| Approach: | They propose a beam search method which is agnostic to the end metric and report results on a variety of metrics. |
| Outcome: | The proposed method is based on a Monte-Carlo Tree Search (MCTS) based method and shows it can be used in language applications. |
A Streaming Approach For Efficient Batched Beam Search (2020.emnlp-main)
Copied to clipboard
| Challenge: | During decoding, candidates terminate or are pruned according to heuristics, a streaming method is used to "refill" the batch after it finishes translating some fraction of the current batch. |
| Approach: | They propose an efficient batching strategy for variable-length decoding on GPU architectures by streamlining the batching process. |
| Outcome: | The proposed method reduces runtime by 71% compared to a fixed-width beam search baseline and 17% compared with a variable-widness baseline while matching baselines’ BLEU. |
On Decoding Strategies for Neural Text Generators (2022.tacl-1)
Copied to clipboard
| Challenge: | a recent study suggests that decoding strategies may be more important than the model architecture itself when generating text from probabilistic models. |
| Approach: | They propose to measure changes in attributes of generated text as a function of decoding strategy and task using human and automatic evaluation. |
| Outcome: | The proposed study shows that decoding strategies do not always transfer across tasks . authors show that the differences in attributes are not always consistent across tasks, they say . |
Exploring Recombination for Efficient Decoding of Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) decoder captures features of entire prediction history . some partial hypotheses with different prefixes will be regarded differently no matter how similar they are . |
| Approach: | They propose a method that uses a n-gram suffix to adapt it to beam search decoding. |
| Outcome: | The proposed method can obtain similar translation quality with a smaller beam size, making it more efficient. |
Improving Beam Search by Removing Monotonic Constraint for Neural Machine Translation (P18-2)
Copied to clipboard
| Challenge: | a beam search algorithm produces monotonic left-to-right order, meaning a hypothesis cannot be revisited . a proposed algorithm allows discarded hypotheses to be recovered in a later step. |
| Approach: | They propose to decode a beam search algorithm that considers multiple hypotheses simultaneously . they propose to maintain all found hypothese a single priority queue and a universal score function . |
| Outcome: | The proposed algorithm improves translations even for high-performance models in English-Japanese translation task. |
Generating Diverse Corrections with Local Beam Search for Grammatical Error Correction (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods of acquiring diverse outputs focus on revising all tokens of a sentence. |
| Approach: | They propose a beam search method to obtain diverse outputs in a local sequence transduction task where most of the tokens in the source and target sentences overlap. |
| Outcome: | The proposed method generates more diverse corrections without losing accuracy in the local sequence transduction task. |