Papers by Anoop Sarkar
Compositionality of Complex Graphemes in the Undeciphered Proto-Elamite Script using Image and Text Embedding Models (2021.findings-acl)
Copied to clipboard
| Challenge: | a language model over sign images produces more interpretable results than a model over text . a new language model is developed to abstract from human annotators . |
| Approach: | They propose a language modeling architecture which operates over sequences of images or over multimodal sequences with associated labels. |
| Outcome: | The proposed language model can interpret signs in the undeciphered proto-Elamite script . it also provides a novel way to abstract away from biases introduced by human annotators. |
Prefix Lexicalization of Synchronous CFGs using Synchronous TAG (P18-1)
Copied to clipboard
| Challenge: | epsilon-free, chain-free synchronous context-free grammars can be converted into weakly equivalent synchronous tree-adjoining grammars (STAGs) this transformation doubles the grammar’s rank and cubes its size, but in practice the size increase is only quadratic. |
| Approach: | They extend Greibach normal form from CFGs to SCFGs and prove new formal properties about SCFG, a formalism with many applications in natural language processing. |
| Outcome: | The proposed grammars achieve asymptotic and empirical speed improvements on a machine translation task. |
Unified Examination of Entity Linking in Absence of Candidate Sets (2024.naacl-short)
Copied to clipboard
| Challenge: | Entity linking systems depend on candidate sets for their performance, but a comprehensive comparative analysis of these systems is lacking. |
| Approach: | They propose a black-box benchmark and a method to evaluate all state-of-the-art entity linking methods. |
| Outcome: | The proposed approach reduces the inference time and memory footprint of some models. |
CipherDAug: Ciphertext based Data Augmentation for Neural Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | a novel data-augmentation technique for neural machine translation is based on a letter substitution cipher . a bijective ciphered text is in effect invisible to modern NLP techniques because of its invariant distributional features . |
| Approach: | They propose a data-augmentation technique for neural machine translation based on ROT-k ciphertexts. |
| Outcome: | The proposed method outperforms existing methods on several datasets by a significant margin. |
Training with Adversaries to Improve Faithfulness of Attention in Neural Machine Translation (2020.aacl-srw)
Copied to clipboard
| Challenge: | Existing approaches to measure faithfulness of neural machine translation models are based on stress tests and a novel objective that rewards faithful behaviour by the model through probability divergence. |
| Approach: | They propose a measure of faithfulness for neural machine translation models based on stress tests and measuring faithfulness based upon how often the model output changes. |
| Outcome: | The proposed objective increases faithfulness without reducing translation quality and can even improve translation quality in some cases. |
Decipherment of Substitution Ciphers with Neural Language Models (D18-1)
Copied to clipboard
| Challenge: | Existing methods for deciphering homophonic substitution ciphers use pre-trained neural LMs. |
| Approach: | They propose a beam search algorithm that scores the entire candidate plaintext at each step of the decipherment using a neural language model. |
| Outcome: | The proposed beam search algorithm improves on challenging ciphers with smaller beam sizes and better error rates than state-of-the-art methods. |
Interrogating the Explanatory Power of Attention in Neural Machine Translation (D19-56)
Copied to clipboard
| Challenge: | Attention models are often used to justify the model’s decision in generating a token but it has not been rigorously established to what extent attention is a reliable source of information in NMT. |
| Approach: | They propose to use attention models to modify crucial aspects of the trained attention model to produce function and content words in the translation process. |
| Outcome: | The proposed models preserve function and content words in the translation process compared to state-of-the-art models. |
Sequence Models for Document Structure Identification in an Undeciphered Script (2022.emnlp-main)
Copied to clipboard
| Challenge: | a systematic analysis of “header” signs in proto-Elamite provides new evidence for their existence . experts have hypothesized that headers are a sign which qualifies transactions . |
| Approach: | They provide unsupervised neural and statistical sequence modeling techniques to identify “header” signs in proto-Elamite, an undeciphered script from 3100-2900 BCE. |
| Outcome: | The authors provide new evidence for the existence of "header" signs in proto-Elamite . they examine which features predict their presence and identify correlations between features and other document properties . |
Decipherment as Regression: Solving Historical Substitution Ciphers by Learning Symbol Recurrence Relations (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for solving substitution ciphers use character-level language models to find key . a Transformer-based causal language model can be used to learn recurrences between characters in a ciphered text . |
| Approach: | They propose a Transformer-based causal language model for decipherment that learns recurrences between characters in a ciphertext. |
| Outcome: | The proposed method achieves strong results on synthetic 1:1 and homophonic ciphers . it learns recurrence relations between cipheries and recovers keys in its self-attention . |
Measuring and Improving Faithfulness of Attention in Neural Machine Translation (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing evidence for faithfulness of neural machine translation models is lacking. |
| Approach: | They propose a novel objective that rewards faithful behaviour by the model through probability divergence and a differentiable objective that can increase faithfulness without reducing the translation quality. |
| Outcome: | The proposed objective increases faithfulness without reducing translation quality and can even improve translation quality in some cases. |
Effectively pretraining a speech translation decoder with Machine Translation data (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to improve the performance of AST systems are based on pretraining the encoder parameters using an ASR model, but using a pretrained MT decoder is not beneficial or improves the results. |
| Approach: | They propose to use an adversarial regularizer to bring the encoder representations of the ASR and NMT tasks closer even though they are in different modalities. |
| Outcome: | The proposed model can be pre-trained using the Automatic Speech Recognition (ASR) task even in different languages and improves in low resource settings. |
SpEL: Structured Prediction for Entity Linking (2023.emnlp-main)
Copied to clipboard
| Challenge: | Entity linking is a key component of structured data creation by linking spans of text to an ontology or knowledge source. |
| Approach: | They propose to use structured prediction for entity linking to classify each input token as an entity and aggregate the token predictions. |
| Outcome: | The proposed system outperforms the state-of-the-art on the commonly used AIDA benchmark dataset for entity linking to Wikipedia. |
Top-down Tree Structured Decoding with Syntactic Connections for Neural Machine Translation and Parsing (D18-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) models are based on sequential decoding or serialisation of structured data into sequence. |
| Approach: | They propose a model that combines sequential encoder with tree-structured decoding augmented with a syntax-aware attention model. |
| Outcome: | The proposed model produces fluent translations with better reordering than previous models. |
Prediction Improves Simultaneous Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Current systems for simultaneous machine translation use an AGENT to control an incremental encoder-decoder model. |
| Approach: | They propose a general-purpose prediction action which predicts future words in the input stream. |
| Outcome: | The proposed agent with prediction has better translation quality and less delay compared to an agent-based system without prediction. |
Translation-based Supervision for Policy Generation in Simultaneous Neural Machine Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to train simultaneous machine translation agents have been used to find the optimal action sequences for translation quality and lag. |
| Approach: | They propose a supervised learning approach that detects minimum reads required for generating target tokens by comparing simultaneous translations against full-sentence translations. |
| Outcome: | The proposed method produces much higher quality translations while minimizing the average lag in simultaneous translation. |
Better Neural Machine Translation by Extracting Linguistic Information from BERT (2021.eacl-main)
Copied to clipboard
| Challenge: | Experimental results show that incorporating linguistic information into neural machine translation models is no more difficult to train than conventional Transformer-based NMT. |
| Approach: | They propose to extract linguistic information from contextual word embeddings instead of point estimates to augment NMT models. |
| Outcome: | The proposed method generalizes better in a variety of training contexts and is no more difficult to train than conventional Transformer-based NMT. |