| Challenge: | Existing methods for deciphering homophonic substitution ciphers use pre-trained neural LMs. |
| Approach: | They propose a beam search algorithm that scores the entire candidate plaintext at each step of the decipherment using a neural language model. |
| Outcome: | The proposed beam search algorithm improves on challenging ciphers with smaller beam sizes and better error rates than state-of-the-art methods. |
Similar Papers
Can Sequence-to-Sequence Models Crack Substitution Ciphers? (2021.acl-long)
Copied to clipboard
| Challenge: | Current methods for deciphering historical ciphers use beam search and a neural language model . but, this approach assumes that the target plaintext language is known . |
| Approach: | They propose an end-to-end multilingual decipherment model that can solve 1:1 substitution ciphers without explicit language identification. |
| Outcome: | The proposed model can decipher text without explicit language identification while still being robust to noise. |
Decipherment as Regression: Solving Historical Substitution Ciphers by Learning Symbol Recurrence Relations (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for solving substitution ciphers use character-level language models to find key . a Transformer-based causal language model can be used to learn recurrences between characters in a ciphered text . |
| Approach: | They propose a Transformer-based causal language model for decipherment that learns recurrences between characters in a ciphertext. |
| Outcome: | The proposed method achieves strong results on synthetic 1:1 and homophonic ciphers . it learns recurrence relations between cipheries and recovers keys in its self-attention . |
Segmenting Numerical Substitution Ciphers (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for deciphering historical substitution ciphers are difficult to crack . cipheries that are not segmented are still difficult to deciphere . |
| Approach: | They propose automatic methods to segment historical substitution ciphers using BPE and unigram language models. |
| Outcome: | The proposed methods achieve an average segmentation error of 2% on 100 monoalphabetic ciphers and 27% on 3 real historical homophonic cipheries. |
Solving Historical Dictionary Codes with a Neural Language Model (2020.emnlp-main)
Copied to clipboard
| Challenge: | a dictionary-based substitution code is common, but no automatic decipherment algorithms exist. |
| Approach: | They propose a decoding lattice and a neural language model to solve word-based substitution codes . they apply their method to letters exchanged between general James Wilkinson and agents of the Spanish Crown . |
| Outcome: | The proposed method decrypts letters written by general James Wilkinson and agents of the Spanish Crown in the late 1700s and early 1800s using a neural language model. |
Exploring Recombination for Efficient Decoding of Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) decoder captures features of entire prediction history . some partial hypotheses with different prefixes will be regarded differently no matter how similar they are . |
| Approach: | They propose a method that uses a n-gram suffix to adapt it to beam search decoding. |
| Outcome: | The proposed method can obtain similar translation quality with a smaller beam size, making it more efficient. |
On Decoding Strategies for Neural Text Generators (2022.tacl-1)
Copied to clipboard
| Challenge: | a recent study suggests that decoding strategies may be more important than the model architecture itself when generating text from probabilistic models. |
| Approach: | They propose to measure changes in attributes of generated text as a function of decoding strategy and task using human and automatic evaluation. |
| Outcome: | The proposed study shows that decoding strategies do not always transfer across tasks . authors show that the differences in attributes are not always consistent across tasks, they say . |
Machine Translation Decoding beyond Beam Search (2021.emnlp-main)
Copied to clipboard
Rémi Leblond, Jean-Baptiste Alayrac, Laurent Sifre, Miruna Pislar, Lespiau Jean-Baptiste, Ioannis Antonoglou, Karen Simonyan, Oriol Vinyals
| Challenge: | a new study examines whether beam search can be replaced by a more powerful metric-driven search technique. |
| Approach: | They propose a beam search method which is agnostic to the end metric and report results on a variety of metrics. |
| Outcome: | The proposed method is based on a Monte-Carlo Tree Search (MCTS) based method and shows it can be used in language applications. |
Simple and Effective Noisy Channel Modeling for Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Previous work on noisy channel modeling relied on latent variable models that incrementally process the source and target sentence. |
| Approach: | They propose to use a standard sequence to sequence model which utilizes the entire source and target sentences to estimate posterior probability of a target sequence y given a source sequence x. |
| Outcome: | The proposed model outperforms direct models on German-English translations by up to 3.2 BLEU on four language pairs. |
Show Some Love to Your n-grams: A Bit of Progress and Stronger n-gram Language Modeling Baselines (N19-1)
Copied to clipboard
| Challenge: | Experimental results show that standalone n-gram models lend themselves as natural choices for resource-lean or morphologically rich languages. |
| Approach: | They run experiments on 50 languages covering all morphological language families to compare n-gram models with lstm models. |
| Outcome: | The proposed extension outperforms an lstm language model on 42 languages while its extension which explicitly injects linguistic knowledge outperformed the character-aware neural model on 8 languages. |
Neural Decipherment via Minimum-Cost Flow: From Ugaritic to Linear B (P19-1)
Copied to clipboard
| Challenge: | Existing methods for decipherment of lost languages are limited by limited data and scarce quantities of ancient text. |
| Approach: | They propose a neural approach for automatic decipherment of lost languages . they use an expressive sequence-to-sequence model to capture character-level correspondences between cognates . |
| Outcome: | The proposed approach improves on the decipherment of Ugaritic and Linear B in ancient Greek . the proposed approach is highly customized for a given language pair and does not generalize to other lost languages. |