Zero-Shot vs. Translation-Based Cross-Lingual Transfer: The Case of Lexical Gaps (2024.naacl-short)
Copied to clipboard
| Challenge: | lexical gaps exist in a variety of domains, such as QA, but they can only be expressed as a combination of words in another language. |
| Approach: | They compare the current performance and long-term viability of two approaches to cross-lingual transfer . they leverage lexical gaps to create a multilingual question answering dataset . |
| Outcome: | The proposed model outperforms zero-shot transfer and machine translation (MT) lexical gaps exist in a variety of domains, including linguistics, linguistic coding, and linguistic analysis. |
Similar Papers
Key ingredients for effective zero-shot cross-lingual knowledge transfer in generative tasks (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing studies have focused on zero-shot cross-lingual transfer . mBERT, mBART and mT5 provide high-quality representations for texts in various languages . |
| Approach: | They propose to use mBART and NLLB-200 to finetune a multilingual pretrained language model on input-output pairs in one language and use it to make task predictions for inputs in other languages. |
| Outcome: | The proposed approach significantly reduces generation in the wrong language with full finetuning and can be competitive in some cases. |
A Simple and Effective Method to Improve Zero-Shot Cross-Lingual Transfer Learning (2022.coling-1)
Copied to clipboard
Kunbo Ding, Weijie Liu, Yuejian Fang, Weiquan Mao, Zhe Zhao, Tao Zhu, Haoyan Liu, Rong Tian, Yiren Chen
| Challenge: | Existing zero-shot cross-lingual transfer methods rely on parallel corpora or bilingual dictionaries . however, its effect is limited by the gap between embedding clusters of different languages . |
| Approach: | They propose Embedding-Push, Attention-Pull, and Robust targets to transfer English embeddings to virtual multilingual embedders without semantic loss. |
| Outcome: | Experimental results show that the proposed method outperforms existing methods on cross-lingual tasks and can achieve a better multilingual alignment. |
To Translate or Not to Translate: A Systematic Investigation of Translation-Based Cross-Lingual Transfer to Low-Resource Languages (2024.naacl-long)
Copied to clipboard
| Challenge: | XLT with multilingual language models is superfluous, says a new study . mBERT, XLM-R and mT5 are effective for cross-lingual transfer, authors say . |
| Approach: | They propose to use multilingual language models to improve cross-lingual transfer (XLT) they propose to add reliable translations to training data for XLT even for non-MT languages . |
| Outcome: | The proposed approaches outperform zero-shot XLT with mLMs, the authors show . the authors believe their findings warrant a broader inclusion of more robust translation-based baselines in XL research. |
Narrowing the Gap between Zero- and Few-shot Machine Translation by Matching Styles (2024.findings-naacl)
Copied to clipboard
Weiting Tan, Haoran Xu, Lingfeng Shen, Shuyue Stella Li, Kenton Murray, Philipp Koehn, Benjamin Van Durme, Yunmo Chen
| Challenge: | Recent work shows that large language models can generalize to machine translation using zero-shot examples with in-context learning. |
| Approach: | They investigate the factors contributing to this gap by matching the writing styles of the target corpus. |
| Outcome: | The proposed methods can be enhanced without the need for parallel demonstration examples. |
Zero-Shot Cross-lingual Semantic Parsing (2022.acl-long)
Copied to clipboard
| Challenge: | Recent work in cross-lingual semantic parsing assumes access to high-quality machine translation systems and word alignment tools. |
| Approach: | They propose a multi-task encoder-decoder model to transfer parsing knowledge to additional languages using only English-logical form paired data and in-domain natural language corpora. |
| Outcome: | The proposed model performs significantly above translation-based baselines and competes with the supervised upper-bound. |
Improving Zero-Shot Multilingual Translation with Universal Representations and Cross-Mapping (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing model structure and training scheme cannot ensure universal representations and cross-mappings because of lacking explicit constraints. |
| Approach: | They propose a state mover’s distance model to model the difference of the representations output by the encoder and a agreement-based training scheme to minimize the proposed distance to learn universal representations. |
| Outcome: | The proposed model can translate between languages unseen during training, i.e., zero-shot translation. |
Analyzing the Evaluation of Cross-Lingual Knowledge Transfer in Multilingual Language Models (2024.eacl-long)
Copied to clipboard
| Challenge: | Recent advances in training multilingual models on large datasets have shown promising results in knowledge transfer across languages. |
| Approach: | They challenge the assumption that high zero-shot performance reflects high cross-lingual ability by introducing more challenging setups involving instances with multiple languages. |
| Outcome: | The proposed model can achieve high performance on multilingual benchmarks and on low-resource languages. |
Towards a Common Understanding of Contributing Factors for Cross-Lingual Transfer in Multilingual Language Models: A Review (2023.acl-long)
Copied to clipboard
| Challenge: | Pre-trained Multilingual Language Models have shown a strong ability to transfer knowledge across languages. |
| Approach: | They examine factors contributing to the ability of MLLMs to perform zero-shot cross-lingual transfer . they identify consensuses among studies with consistent findings and resolve conflicts . |
| Outcome: | The authors outline and discuss factors that contribute to the ability of MLLMs to perform zero-shot cross-lingual transfer. |
Make the Best of Cross-lingual Transfer: Evidence from POS Tagging with over 100 Languages (2022.acl-long)
Copied to clipboard
| Challenge: | Existing studies on cross-lingual generalisability of large pre-trained models use English training data and test data in unseen languages. |
| Approach: | They propose to use multilingual pre-trained models to model cross-lingual transfer in a selection of target languages. |
| Outcome: | The proposed model can be used to improve cross-lingual transfer performance in low-resource languages with no labeled training data. |
Why Does Zero-Shot Cross-Lingual Generation Fail? An Explanation and a Solution (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on cross-lingual transferability of multilingual LMs show that they can perform tasks in low-resource languages. |
| Approach: | They propose a method to regularize the model from learning language invariant representations and a way to select model checkpoints without a development set in the target language. |
| Outcome: | The proposed method reduces the accidental translation problem by 68% and improves the ROUGE-L score by 1.5 on average. |