Papers by Pooya Moradi
Training with Adversaries to Improve Faithfulness of Attention in Neural Machine Translation (2020.aacl-srw)
Copied to clipboard
| Challenge: | Existing approaches to measure faithfulness of neural machine translation models are based on stress tests and a novel objective that rewards faithful behaviour by the model through probability divergence. |
| Approach: | They propose a measure of faithfulness for neural machine translation models based on stress tests and measuring faithfulness based upon how often the model output changes. |
| Outcome: | The proposed objective increases faithfulness without reducing translation quality and can even improve translation quality in some cases. |
Interrogating the Explanatory Power of Attention in Neural Machine Translation (D19-56)
Copied to clipboard
| Challenge: | Attention models are often used to justify the model’s decision in generating a token but it has not been rigorously established to what extent attention is a reliable source of information in NMT. |
| Approach: | They propose to use attention models to modify crucial aspects of the trained attention model to produce function and content words in the translation process. |
| Outcome: | The proposed models preserve function and content words in the translation process compared to state-of-the-art models. |
Measuring and Improving Faithfulness of Attention in Neural Machine Translation (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing evidence for faithfulness of neural machine translation models is lacking. |
| Approach: | They propose a novel objective that rewards faithful behaviour by the model through probability divergence and a differentiable objective that can increase faithfulness without reducing the translation quality. |
| Outcome: | The proposed objective increases faithfulness without reducing translation quality and can even improve translation quality in some cases. |