Pre-tokenization of Multi-word Expressions in Cross-lingual Word Embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Multi-Word Expressions (MWEs) are common in every language, but they are not translated by cross-lingual word embeddings. |
| Approach: | They propose a method for word translation of Multi-Word Expressions (MWEs) they compile lists of MWEs in each language and tokenize them as single tokens before training word embeddings. |
| Outcome: | The proposed method can translate multi-word expressions to and from English in 10 languages. |
Similar Papers
Unsupervised Multilingual Word Embeddings (D18-1)
Copied to clipboard
| Challenge: | Prior art for learning UMWEs relies on a number of independently trained UBWEs to obtain multilingual embeddings. |
| Approach: | They propose a fully unsupervised framework that exploits the relations between all language pairs to learn multilingual embeddings without cross-lingual supervision. |
| Outcome: | The proposed framework outperforms supervised approaches on multilingual word translation and cross-lingual word similarity and beats a number of other approaches trained with cross-linguistic resources. |
MultiMWE: Building a Multi-lingual Multi-Word Expression (MWE) Parallel Corpora (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing bilingual or multi-lingual MWE corpora are limited for multilingual use . only 871 pairs of English-German MWEs are available for research . |
| Approach: | They present a collection of bilingual and multi-lingual MWEs extracted from parallel corpora. |
| Outcome: | The available bilingual or multi-lingual MWE corpus is very limited . the collection is a small collection of 871 pairs of English-German MWEs . |
KIT-Multi: A Translation-Oriented Multilingual Embedding Corpus (L18-1)
Copied to clipboard
| Challenge: | Cross-lingual word embeddings are representations of words across languages in a shared continuous vector space. |
| Approach: | They propose a multilingual word embedding corpus which is acquired by neural machine translation and is based on monolingual data. |
| Outcome: | The proposed method is competitive with existing methods but on the cross-lingual document classification task, it obtains the best figures. |
CoAM: Corpus of All-Type Multiword Expressions (2025.acl-long)
Copied to clipboard
Yusuke Ide, Joshua Tanner, Adam Nohejl, Jacob Hoffman, Justin Vasselli, Hidetaka Kamigaito, Taro Watanabe
| Challenge: | Existing datasets for multiword expressions are inconsistently annotated, limited to a single type of MWE, or limited in size. |
| Approach: | They propose to use a new interface to generate MWE annotations for the first time in a dataset of MWE identification. |
| Outcome: | The proposed model outperforms existing models on the DiMSUM dataset. |
Multilingual Word Segmentation: Training Many Language-Specific Tokenizers Smoothly Thanks to the Universal Dependencies Corpus (L18-1)
Copied to clipboard
| Challenge: | Towards language scalability, major progress has been achieved in multilingual language technology in recent years. |
| Approach: | They propose a tokenizer that can be trained from any Universal Dependencies corpus dataset . they argue that tokenization should be seen as a supervised task and scalability requires a software engineering process across languages. |
| Outcome: | The proposed tokenizer can be trained from any dataset in the corpus UD2 . the proposed software tool relies on elephant to perform the training . |
How to (Properly) Evaluate Cross-Lingual Word Embeddings: On Strong Baselines, Comparative Analyses, and Some Misconceptions (P19-1)
Copied to clipboard
| Challenge: | Cross-lingual word embeddings (CLEs) are used for downstream NLP tasks . CLEs are based on bilingual lexicon induction (BLI) evaluations vary greatly, hindering ability to interpret performance and properties of different CLE models. |
| Approach: | They evaluate CLE models for a large number of language pairs on bilingual lexicon induction and three downstream tasks. |
| Outcome: | The proposed model performance is based on supervised and unsupervised models on bilingual lexicon induction and three downstream tasks. |
Beyond Shared Vocabulary: Increasing Representational Word Similarities across Languages for Multilingual Machine Translation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Using a shared vocabulary is common practice in multilingual machine translation . however, when words overlap is small, e.g., using different writing systems, knowledge transfer is inhibited . |
| Approach: | They propose a re-parameterized method for building word embeddings using word equivalence classes and graph networks to fuse word embeds across languages. |
| Outcome: | The proposed method achieves evident BLEU improvements on high- and low-resource MNMT scenarios. |
A Multi-dimensional Evaluation of Tokenizer-free Multilingual Pretrained Models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent work on tokenizer-free models shows promising results in cross-lingual transfer . previous work focused on reporting accuracy on a limited set of tasks and data settings . |
| Approach: | They compare tokenizer-free and subword-based models using various dimensions . they find subword models are still the most practical choice in many settings . |
| Outcome: | The proposed model improves cross-lingual transfer and reduces engineering overhead. |
WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to train large pretrained language models require more computational resources and are expensive to train in other languages. |
| Approach: | They propose a method to transfer pretrained language models to new languages using subword-based tokenization and embeddings. |
| Outcome: | The proposed method outperforms existing methods on low-resource languages and makes training large models more accessible and less damaging to the environment. |
Analyzing the Limitations of Cross-lingual Word Embedding Mappings (P19-1)
Copied to clipboard
| Challenge: | Existing methods for cross-lingual word embeddings have limited results . existing methods require little or no cross-linguistic signal to work . |
| Approach: | They compare offline mapping methods to an extension of skip-gram that jointly learns both embedding spaces. |
| Outcome: | The proposed method yields more isomorphic embeddings, is less sensitive to hubness, and achieves stronger results in bilingual lexicon induction. |