Papers by Sebastian Donoso
ALBETO and DistilBETO: Lightweight Spanish Language Models (2022.lrec-1)
Copied to clipboard
| Challenge: | Recent advances in pre-trained language models have made them more popular . however, there are still limited versions of these models for other languages . |
| Approach: | They present ALBETO and DistilBETO which are versions of ALBERT and DistillBERT pre-trained exclusively on Spanish corpora. |
| Outcome: | The proposed models outperform BETO and ALBERT on Spanish datasets . the models outpace BETO on MLDoc, PAWS-X, XNLI, MLQA, SQAC and XQuAD datasets. |