Lost in Evaluation: Misleading Benchmarks for Bilingual Dictionary Induction (D19-1)
Copied to clipboard
| Challenge: | a quarter of the data consists of proper nouns, which can be hardly indicative of BDI performance, and there are pervasive gaps in the gold-standard targets. |
| Approach: | They examine the composition and quality of test sets for five different languages . they suggest future research avoids drawing conclusions from quantitative results . |
| Outcome: | The results show that a quarter of the data consists of proper nouns, which can be hardly indicative of BDI performance, and there are gaps in the gold-standard targets. |
Similar Papers
Combining Word Embeddings with Bilingual Orthography Embeddings for Bilingual Dictionary Induction (2020.coling-main)
Copied to clipboard
| Challenge: | Bilingual dictionary induction (BDI) is a task of finding target language translations of source language words. |
| Approach: | They propose to use bilingual orthography Embeddings to enrich BWE-based BDI with transliteration information to make a decision on which information source is more reliable for a particular word pair. |
| Outcome: | The proposed system improves on English-Russian BDI and shows that it can be built with only weak bilingual signals and even without any bilingual signal. |
Evaluating bilingual word embeddings on the long tail (N18-2)
Copied to clipboard
| Challenge: | Bilingual word embeddings are useful for bilingual lexicon induction, but they focus on frequent words in general domains. |
| Approach: | They propose to evaluate bilingual word embeddings on rare words in different domains . they propose to use a multilingual dataset to build and combine BWEs based on a single word . |
| Outcome: | The proposed evaluations show that state-of-the-art methods fail on rare words . the proposed evaluation is based on a gold standard dataset and code . |
How Lexical is Bilingual Lexicon Induction? (2024.findings-naacl)
Copied to clipboard
| Challenge: | lexical variation and low-resource settings make it difficult to learn in low-level settings. |
| Approach: | They propose to incorporate additional lexical information into the retrieve-and-rank approach to improve lexicon induction. |
| Outcome: | The proposed approach improves on XLING by an average of 2% across all language pairs. |
On the Limitations of Unsupervised Bilingual Dictionary Induction (P18-1)
Copied to clipboard
| Challenge: | Unsupervised machine translation does not require cross-lingual supervision, whether a dictionary, translations, or comparable corpora. |
| Approach: | They propose an adversarial, unsupervised cross-lingual word embedding technique for bilingual dictionary induction that exploits a weak supervision signal from identical words. |
| Outcome: | The proposed model relies heavily on an adversarial, unsupervised cross-lingual word embedding technique for bilingual dictionary induction. |
How to (Properly) Evaluate Cross-Lingual Word Embeddings: On Strong Baselines, Comparative Analyses, and Some Misconceptions (P19-1)
Copied to clipboard
| Challenge: | Cross-lingual word embeddings (CLEs) are used for downstream NLP tasks . CLEs are based on bilingual lexicon induction (BLI) evaluations vary greatly, hindering ability to interpret performance and properties of different CLE models. |
| Approach: | They evaluate CLE models for a large number of language pairs on bilingual lexicon induction and three downstream tasks. |
| Outcome: | The proposed model performance is based on supervised and unsupervised models on bilingual lexicon induction and three downstream tasks. |
Should All Cross-Lingual Embeddings Speak English? (2020.acl-main)
Copied to clipboard
| Challenge: | lexicon induction evaluation dictionaries are mostly between English and another language, and the English hub is selected by default as the hub . lexiconic embeddings are often learned with a two-step process, whether under bilingual or multilingual settings. |
| Approach: | They propose to use English as the hub language for lexicon induction evaluation . they also expand a standard English-centered evaluation dictionary collection to include all language pairs . |
| Outcome: | The proposed method can significantly improve lexicon induction performance over multiple languages. |
Orthographic Features for Bilingual Lexicon Induction (P18-2)
Copied to clipboard
| Challenge: | Recent embedding-based methods do not take advantage of orthographic features, such as edit distance, which can be helpful for pairs of related languages. |
| Approach: | They propose to use orthographic features to integrate orthographic induction into embedding methods . they use document-aligned data instead of a seed dictionary to learn bilingual embedds . |
| Outcome: | This work extends embedding-based methods to incorporate orthographic features . it shows that the methods can learn bilingual embeddables in low-resource languages . |
Bilingual Lexicon Induction via Unsupervised Bitext Construction and Word Alignment (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for bilingual lexicon induction are linear and require simplifying assumptions. |
| Approach: | They propose methods that combine unsupervised bitext mining and unsupervised word alignment to produce higher quality lexicons. |
| Outcome: | The proposed method outperforms the state-of-the-art on the BUCC 2020 task by 14 F1 points . further analysis suggests they are comparable quality . |
A Simple and Effective Approach to Robust Unsupervised Bilingual Dictionary Induction (2020.coling-main)
Copied to clipboard
| Challenge: | Recent work has questioned the robustness of unsupervised bilingual dictionary induction methods on distant language pairs. |
| Approach: | They propose an iterative dimension reduction method to bridge this gap . they propose a method that initializes and self-learning and inducing a dictionary . |
| Outcome: | The proposed method achieves 13.64 55.53% accuracy between English and four distant languages. |
Why Overfitting Isn’t Always Bad: Retrofitting Cross-Lingual Word Embeddings to Dictionaries (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies only evaluate cross-lingual word embeddings on bilingual lexicon induction (BLI) however, underfitting can hinder generalization to other downstream tasks. |
| Approach: | They retrofit cross-lingual word embeddings to the training dictionary and a synthetic dictionary to improve their results. |
| Outcome: | The proposed method improves accuracy on two downstream tasks, despite underfitting the training dictionary. |