Papers with VecMap
Combining Static and Contextualised Multilingual Embeddings (2022.findings-acl)
Copied to clipboard
| Challenge: | Static embeddings are less expressive than contextual language models, but can be more straightforwardly aligned across multiple languages. |
| Approach: | They extract static embeddings for 40 languages from XLM-R and validate them with cross-lingual word retrieval and then align them using VecMap. |
| Outcome: | The proposed approach improves multilingual representations by leveraging static embeddings and a pre-training code. |
Improving Bilingual Lexicon Induction with Cross-Encoder Reranking (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Current methods for bilingual lexicon induction rely on the induction of cross-lingual word embeddings (CLWEs) such as VecMap or mPLMs are not available for multilingual NLP. |
| Approach: | They propose a semi-supervised post-hoc reranking method which combines cross-lingual lexical knowledge from multilingual pretrained language models with original CLWEs. |
| Outcome: | The proposed method outperforms existing methods on two standard benchmarks spanning a wide spectrum of languages and is robust to different CLWEs. |
Identifying Elements Essential for BERT’s Multilinguality (2020.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual BERT (mBERT) does not use any crosslingual signal during training. |
| Approach: | They propose a multilingual pretraining setup that modifies the masking strategy using VecMap to allow for fast experimentation. |
| Outcome: | The proposed setup with pretrained models with three languages shows that it works well. |