Papers by Clément Lefebvre
LuxemBERT: Simple and Practical Data Augmentation in Language Model Pre-Training for Luxembourgish (2022.lrec-1)
Copied to clipboard
Cedric Lothritz, Bertrand Lebichot, Kevin Allix, Lisa Veiber, Tegawende Bissyande, Jacques Klein, Andrey Boytsov, Clément Lefebvre, Anne Goujon
| Challenge: | Pre-trained Language Models such as BERT are ubiquitous in NLP but are scarce for low-resource languages such as Luxembourgish. |
| Approach: | They propose a BERT model for Luxembourgish language that they use to augment pre-training datasets by partially translating text data from a closely related language. |
| Outcome: | The proposed model outperforms the baseline model and the mBERT model in Luxembourgish. |