Papers by Lukasz Kaiser
The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation (P18-1)
Copied to clipboard
Mia Xu Chen, Orhan Firat, Ankur Bapna, Melvin Johnson, Wolfgang Macherey, George Foster, Llion Jones, Mike Schuster, Noam Shazeer, Niki Parmar, Ashish Vaswani, Jakob Uszkoreit, Lukasz Kaiser, Zhifeng Chen, Yonghui Wu, Macduff Hughes
| Challenge: | In recent years, the emergence of seq2seq models has revolutionized the field of machine translation by replacing traditional phrase-based approaches with neural machine translation (NMT) systems based on the encoder-decoder paradigm. |
| Approach: | They propose to use a convolutional seq2seq model to combine the strengths of the two approaches. |
| Outcome: | The proposed architectures outperform the existing models on the WMT’14 benchmark dataset. |
Hierarchical Transformers Are More Efficient Language Models (2022.findings-naacl)
Copied to clipboard
Piotr Nawrot, Szymon Tworkowski, Michał Tyrolski, Lukasz Kaiser, Yuhuai Wu, Christian Szegedy, Henryk Michalewski
| Challenge: | Transformers are impressive but inefficient and costly, which limits their applications and accessibility. |
| Approach: | They first use different ways to downsample and upsamplify activations in Transformers to make them hierarchical. |
| Outcome: | The proposed model outperforms Transformers on the ImageNet32 and enwik8 benchmarks. |