Papers by Julien Launay
PAGnol: An Extra-Large French Generative Model (2022.lrec-1)
Copied to clipboard
Julien Launay, E.l. Tommasone, Baptiste Pannier, François Boniface, Amélie Chatelain, Alessandro Cappelli, Iacopo Poli, Djamé Seddah
| Challenge: | a growing number of pre-trained language models are available in many different languages. |
| Approach: | They propose a French-language GPT model with scaling laws to train it efficiently . they evaluate the models on discriminative and generative tasks in French . |
| Outcome: | The proposed model trains with the same computational budget as CamemBERT, a model 13 times smaller. |
What Language Model to Train if You Have One Million GPU Hours? (2022.findings-emnlp)
Copied to clipboard
Teven Le Scao, Thomas Wang, Daniel Hesslow, Stas Bekman, M Saiful Bari, Stella Biderman, Hady Elsahar, Niklas Muennighoff, Jason Phang, Ofir Press, Colin Raffel, Victor Sanh, Sheng Shen, Lintang Sutawika, Jaesung Tae, Zheng Xin Yong, Julien Launay, Iz Beltagy
| Challenge: | Recent years have seen the advent of large language models characterized by emergent capabilities arising from sheer scale alone. |
| Approach: | They propose to use a multilingual model to compare performance to the English-only model by ablation at the billion-parameter scale. |
| Outcome: | The proposed model is based on a multilingual model and its performance against the English-only model. |