Papers by Sajad Norouzi
Code Generation from Natural Language with Less Prior Knowledge and More Monolingual Data (2021.acl-short)
Copied to clipboard
| Challenge: | a generic transformer-based model can achieve competitive performance with minimal code-generation-specific inductive bias design. |
| Approach: | They investigate whether a generic transformer-based seq2seq model can achieve competitive performance with minimal code-generation-specific inductive bias design. |
| Outcome: | The proposed model achieves 81.03% exact match accuracy on Django and 32.57 BLEU score on CoNaLa. |
DiMS: Distilling Multiple Steps of Iterative Non-Autoregressive Transformers for Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Neural machine translation models follow an autoregressive decoding strategy, generating the target sentence one token at a time. |
| Approach: | They propose a distillation technique that reduces the number of decoding steps required to reach a certain translation quality. |
| Outcome: | The proposed model achieves 7.8 and 12.9 BLEU points improvements in translation accuracy on distilled and raw versions of WMT’14 De-En. |