Are Girls Neko or Shōjo? Cross-Lingual Alignment of Non-Isomorphic Embeddings with Iterative Normalization (P19-1)
Copied to clipboard
| Challenge: | Cross-lingual word embeddings (CLWE) are used to perform multilingual natural language processing tasks. |
| Approach: | They propose a method that transforms monolingual embeddings to make orthogonal alignment easier by simultaneously enforcing that (1) individual word vectors are unit length, and (2) each language’s average vector is zero. |
| Outcome: | The proposed method improves translation accuracy of three CLWE methods, with the largest improvement observed on English-Japanese (2% to 44% test accuracy). |
Similar Papers
Cross-Lingual Word Embedding Refinement by ℓ1 Norm Optimisation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for building high-quality CLWEs learn mappings that minimise the l2 norm loss function but this optimisation objective has been shown to be sensitive to outliers. |
| Approach: | They propose a simple post-processing step to improve cross-lingual word embeddings using the Manhattan norm goodness-of-fit criterion. |
| Outcome: | The proposed approach outperforms four state-of-the-art baselines in bilingual lexicon induction and cross-lingual transfer tasks. |
Analyzing the Limitations of Cross-lingual Word Embedding Mappings (P19-1)
Copied to clipboard
| Challenge: | Existing methods for cross-lingual word embeddings have limited results . existing methods require little or no cross-linguistic signal to work . |
| Approach: | They compare offline mapping methods to an extension of skip-gram that jointly learns both embedding spaces. |
| Outcome: | The proposed method yields more isomorphic embeddings, is less sensitive to hubness, and achieves stronger results in bilingual lexicon induction. |
Non-Linearity in Mapping Based Cross-Lingual Word Embeddings (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing work on cross-lingual word embeddings rely on linear mappings, but this assumption is not true for all language pairs. |
| Approach: | They propose a non-linear mapping approach which can find non-linesar relationships between languages by kernel Canonical Correlation Analysis. |
| Outcome: | The proposed approach improves on five language pairs on supervised and self-learning scenarios. |
Hierarchical Mapping for Crosslingual Word Embedding Alignment (2020.tacl-1)
Copied to clipboard
| Challenge: | Existing strategies that map word embeddings into a crosslingual space are biased towards the choice of the pivot language. |
| Approach: | They propose to map any two languages into a different middle space by learning mappings across languages in a hierarchical way. |
| Outcome: | The proposed strategy significantly improves vocabulary induction scores in all existing benchmarks and in a new non-English–centered benchmark. |
Improving Cross-Lingual Word Embeddings by Meeting in the Middle (D18-1)
Copied to clipboard
| Challenge: | Cross-lingual word embeddings are becoming increasingly important in multilingual NLP. |
| Approach: | They propose to apply an additional transformation after initial alignment to align two disjoint monolingual vector spaces. |
| Outcome: | The proposed approach outperforms state-of-the-art models in monolingual and cross-lingual evaluation tasks. |
LNMap: Departures from Isomorphic Assumption in Bilingual Lexicon Induction Through Non-Linear Mapping in Latent Space (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for bilingual lexicon induction are mapping-based, but they do not hold for closely related languages. |
| Approach: | They propose a semi-supervised method to learn cross-lingual word embeddings for BLI using a linear mapping function and a latent space of two independently trained autoencoders. |
| Outcome: | The proposed method outperforms existing models on 15 different language pairs on both directions. |
Do We Really Need Fully Unsupervised Cross-Lingual Embeddings? (D19-1)
Copied to clipboard
| Challenge: | a series of bilingual lexicon induction experiments with 15 diverse languages (210 language pairs) show that fully unsupervised CLWE methods fail for a large number of language pairs. |
| Approach: | They propose to use fully unsupervised approaches to project monolingual embeddings into a shared cross-lingual space without any cross-linguistic signal. |
| Outcome: | The proposed methods fail for a large number of language pairs, but never surpass weakly supervised methods. |
Unsupervised Joint Training of Bilingual Word Embeddings (P19-1)
Copied to clipboard
| Challenge: | Existing methods for unsupervised bilingual word embeddings are limited by the dissimilarity between the word embedded spaces. |
| Approach: | They propose a method that trains unsupervised bilingual word embeddings jointly on parallel data generated through unsupervised machine translation. |
| Outcome: | The proposed method outperforms unsupervised mapped bilingual word embeddings in cross-lingual NLP tasks. |
Understanding Cross-Lingual Alignment—A Survey (2024.findings-acl)
Copied to clipboard
| Challenge: | Cross-lingual alignment is the meaningful similarity of representations across languages in multilingual language models. |
| Approach: | They propose a taxonomy of methods to improve cross-lingual alignment . they argue that an effective trade-off between language-neutral and language-specific information is key . |
| Outcome: | The proposed methods can be applied to encoder models and encoder-decoder-only models . they show that language-neutral and language-specific information is key . |
How to (Properly) Evaluate Cross-Lingual Word Embeddings: On Strong Baselines, Comparative Analyses, and Some Misconceptions (P19-1)
Copied to clipboard
| Challenge: | Cross-lingual word embeddings (CLEs) are used for downstream NLP tasks . CLEs are based on bilingual lexicon induction (BLI) evaluations vary greatly, hindering ability to interpret performance and properties of different CLE models. |
| Approach: | They evaluate CLE models for a large number of language pairs on bilingual lexicon induction and three downstream tasks. |
| Outcome: | The proposed model performance is based on supervised and unsupervised models on bilingual lexicon induction and three downstream tasks. |