| Challenge: | Current methods for deciphering historical ciphers use beam search and a neural language model . but, this approach assumes that the target plaintext language is known . |
| Approach: | They propose an end-to-end multilingual decipherment model that can solve 1:1 substitution ciphers without explicit language identification. |
| Outcome: | The proposed model can decipher text without explicit language identification while still being robust to noise. |
Similar Papers
Decipherment of Substitution Ciphers with Neural Language Models (D18-1)
Copied to clipboard
| Challenge: | Existing methods for deciphering homophonic substitution ciphers use pre-trained neural LMs. |
| Approach: | They propose a beam search algorithm that scores the entire candidate plaintext at each step of the decipherment using a neural language model. |
| Outcome: | The proposed beam search algorithm improves on challenging ciphers with smaller beam sizes and better error rates than state-of-the-art methods. |
Solving Historical Dictionary Codes with a Neural Language Model (2020.emnlp-main)
Copied to clipboard
| Challenge: | a dictionary-based substitution code is common, but no automatic decipherment algorithms exist. |
| Approach: | They propose a decoding lattice and a neural language model to solve word-based substitution codes . they apply their method to letters exchanged between general James Wilkinson and agents of the Spanish Crown . |
| Outcome: | The proposed method decrypts letters written by general James Wilkinson and agents of the Spanish Crown in the late 1700s and early 1800s using a neural language model. |
Simple and Effective Noisy Channel Modeling for Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Previous work on noisy channel modeling relied on latent variable models that incrementally process the source and target sentence. |
| Approach: | They propose to use a standard sequence to sequence model which utilizes the entire source and target sentences to estimate posterior probability of a target sequence y given a source sequence x. |
| Outcome: | The proposed model outperforms direct models on German-English translations by up to 3.2 BLEU on four language pairs. |
Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Tasks (2021.naacl-main)
Copied to clipboard
| Challenge: | ESPnet framework exploits compositionality to learn searchable hidden representations at intermediate stages of a sequence model using decomposed sub-tasks. |
| Approach: | They propose a framework that exploits compositionality to learn searchable hidden representations at intermediate stages of a sequence model using decomposed sub-tasks. |
| Outcome: | The proposed framework outperforms the state-of-the-art on speech translation tasks by +6 and +3 BLEU on the two test sets of Fisher-CallHome and +4 BLUE on the English-German and English-French test sets. |
Machine Translation Decoding beyond Beam Search (2021.emnlp-main)
Copied to clipboard
Rémi Leblond, Jean-Baptiste Alayrac, Laurent Sifre, Miruna Pislar, Lespiau Jean-Baptiste, Ioannis Antonoglou, Karen Simonyan, Oriol Vinyals
| Challenge: | a new study examines whether beam search can be replaced by a more powerful metric-driven search technique. |
| Approach: | They propose a beam search method which is agnostic to the end metric and report results on a variety of metrics. |
| Outcome: | The proposed method is based on a Monte-Carlo Tree Search (MCTS) based method and shows it can be used in language applications. |
Exploring Recombination for Efficient Decoding of Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) decoder captures features of entire prediction history . some partial hypotheses with different prefixes will be regarded differently no matter how similar they are . |
| Approach: | They propose a method that uses a n-gram suffix to adapt it to beam search decoding. |
| Outcome: | The proposed method can obtain similar translation quality with a smaller beam size, making it more efficient. |
Comparison of Diverse Decoding Methods from Conditional Language Models (P19-1)
Copied to clipboard
| Challenge: | Conditional language models can generate a diverse set of outputs, but for open-ended tasks, beam search is ill-suited to generating a set of diverse sequences. |
| Approach: | They propose a method where we over-sample candidates and use clustering to remove similar sequences to achieve high diversity without sacrificing quality. |
| Outcome: | The proposed method over-samples candidates and removes similar sequences to achieve high diversity without sacrificing quality. |
GEC-DePenD: Non-Autoregressive Grammatical Error Correction with Decoupled Permutation and Decoding (2023.acl-long)
Copied to clipboard
| Challenge: | grammatical error correction is an important NLP task that is usually solved with autoregressive sequence-to-sequence models. |
| Approach: | They propose a non-autoregressive approach to grammatical error correction that decouples a permutation network and a decoder network that fills in specific tokens. |
| Outcome: | The proposed approach improves over previously known non-autoregressive methods and reaches the level of autoregressive approaches that do not use language-specific synthetic data generation methods. |
Neural Decipherment via Minimum-Cost Flow: From Ugaritic to Linear B (P19-1)
Copied to clipboard
| Challenge: | Existing methods for decipherment of lost languages are limited by limited data and scarce quantities of ancient text. |
| Approach: | They propose a neural approach for automatic decipherment of lost languages . they use an expressive sequence-to-sequence model to capture character-level correspondences between cognates . |
| Outcome: | The proposed approach improves on the decipherment of Ugaritic and Linear B in ancient Greek . the proposed approach is highly customized for a given language pair and does not generalize to other lost languages. |
If beam search is the answer, what was the question? (2020.emnlp-main)
Copied to clipboard
| Challenge: | surprisingly, beam search results on language generation tasks are low-quality . despite its high error rate, beam searches can be used to decode models with high probability . |
| Approach: | They frame beam search as the exact solution to a different decoding objective . they propose a set of decoding objectives that explicitly enforce this property . |
| Outcome: | The proposed method enforces uniform information density in text, a property motivated by cognitive science. |