Why Overfitting Isn’t Always Bad: Retrofitting Cross-Lingual Word Embeddings to Dictionaries (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies only evaluate cross-lingual word embeddings on bilingual lexicon induction (BLI) however, underfitting can hinder generalization to other downstream tasks. |
| Approach: | They retrofit cross-lingual word embeddings to the training dictionary and a synthetic dictionary to improve their results. |
| Outcome: | The proposed method improves accuracy on two downstream tasks, despite underfitting the training dictionary. |
Similar Papers
How to (Properly) Evaluate Cross-Lingual Word Embeddings: On Strong Baselines, Comparative Analyses, and Some Misconceptions (P19-1)
Copied to clipboard
| Challenge: | Cross-lingual word embeddings (CLEs) are used for downstream NLP tasks . CLEs are based on bilingual lexicon induction (BLI) evaluations vary greatly, hindering ability to interpret performance and properties of different CLE models. |
| Approach: | They evaluate CLE models for a large number of language pairs on bilingual lexicon induction and three downstream tasks. |
| Outcome: | The proposed model performance is based on supervised and unsupervised models on bilingual lexicon induction and three downstream tasks. |
Do We Really Need Fully Unsupervised Cross-Lingual Embeddings? (D19-1)
Copied to clipboard
| Challenge: | a series of bilingual lexicon induction experiments with 15 diverse languages (210 language pairs) show that fully unsupervised CLWE methods fail for a large number of language pairs. |
| Approach: | They propose to use fully unsupervised approaches to project monolingual embeddings into a shared cross-lingual space without any cross-linguistic signal. |
| Outcome: | The proposed methods fail for a large number of language pairs, but never surpass weakly supervised methods. |
Analyzing the Limitations of Cross-lingual Word Embedding Mappings (P19-1)
Copied to clipboard
| Challenge: | Existing methods for cross-lingual word embeddings have limited results . existing methods require little or no cross-linguistic signal to work . |
| Approach: | They compare offline mapping methods to an extension of skip-gram that jointly learns both embedding spaces. |
| Outcome: | The proposed method yields more isomorphic embeddings, is less sensitive to hubness, and achieves stronger results in bilingual lexicon induction. |
Improving Bilingual Lexicon Induction with Cross-Encoder Reranking (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Current methods for bilingual lexicon induction rely on the induction of cross-lingual word embeddings (CLWEs) such as VecMap or mPLMs are not available for multilingual NLP. |
| Approach: | They propose a semi-supervised post-hoc reranking method which combines cross-lingual lexical knowledge from multilingual pretrained language models with original CLWEs. |
| Outcome: | The proposed method outperforms existing methods on two standard benchmarks spanning a wide spectrum of languages and is robust to different CLWEs. |
Lost in Evaluation: Misleading Benchmarks for Bilingual Dictionary Induction (D19-1)
Copied to clipboard
| Challenge: | a quarter of the data consists of proper nouns, which can be hardly indicative of BDI performance, and there are pervasive gaps in the gold-standard targets. |
| Approach: | They examine the composition and quality of test sets for five different languages . they suggest future research avoids drawing conclusions from quantitative results . |
| Outcome: | The results show that a quarter of the data consists of proper nouns, which can be hardly indicative of BDI performance, and there are gaps in the gold-standard targets. |
Are Girls Neko or Shōjo? Cross-Lingual Alignment of Non-Isomorphic Embeddings with Iterative Normalization (P19-1)
Copied to clipboard
| Challenge: | Cross-lingual word embeddings (CLWE) are used to perform multilingual natural language processing tasks. |
| Approach: | They propose a method that transforms monolingual embeddings to make orthogonal alignment easier by simultaneously enforcing that (1) individual word vectors are unit length, and (2) each language’s average vector is zero. |
| Outcome: | The proposed method improves translation accuracy of three CLWE methods, with the largest improvement observed on English-Japanese (2% to 44% test accuracy). |
Should All Cross-Lingual Embeddings Speak English? (2020.acl-main)
Copied to clipboard
| Challenge: | lexicon induction evaluation dictionaries are mostly between English and another language, and the English hub is selected by default as the hub . lexiconic embeddings are often learned with a two-step process, whether under bilingual or multilingual settings. |
| Approach: | They propose to use English as the hub language for lexicon induction evaluation . they also expand a standard English-centered evaluation dictionary collection to include all language pairs . |
| Outcome: | The proposed method can significantly improve lexicon induction performance over multiple languages. |
Combining Static Word Embeddings and Contextual Representations for Bilingual Lexicon Induction (2021.findings-acl)
Copied to clipboard
| Challenge: | Bilingual Lexicon Induction (BLI) aims to map words in one language to their translations in another. |
| Approach: | They propose a mechanism to combine static word embeddings and contextual representations to utilize the advantages of both paradigms. |
| Outcome: | The proposed method improves performance on supervised and unsupervised BLI benchmarks on all language pairs by average improving 3.2 points over baselines. |
Beyond Offline Mapping: Learning Cross-lingual Word Embeddings through Context Anchoring (2021.acl-long)
Copied to clipboard
| Challenge: | Recent research on cross-lingual word embeddings has been dominated by unsupervised mapping approaches that align monolingual embedders. |
| Approach: | They propose an unsupervised mapping approach that fixes fixed embeddings and learns new ones for the source language that are aligned with them. |
| Outcome: | The proposed method outperforms conventional mapping methods on bilingual lexicon induction and obtains competitive results in the downstream XNLI task. |
On the Limitations of Unsupervised Bilingual Dictionary Induction (P18-1)
Copied to clipboard
| Challenge: | Unsupervised machine translation does not require cross-lingual supervision, whether a dictionary, translations, or comparable corpora. |
| Approach: | They propose an adversarial, unsupervised cross-lingual word embedding technique for bilingual dictionary induction that exploits a weak supervision signal from identical words. |
| Outcome: | The proposed model relies heavily on an adversarial, unsupervised cross-lingual word embedding technique for bilingual dictionary induction. |