Improving Cross-lingual Transfer through Subtree-aware Word Reordering (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that multilingual language models are not effective when dealing with less-represented languages. |
| Approach: | They propose a powerful reordering method that learns word-order patterns conditioned on the syntactic context from a small amount of annotated data. |
| Outcome: | The proposed method outperforms baselines on a variety of tasks and is effective in both zero-shot and few-shot scenarios. |
Similar Papers
Low-Resource Syntactic Transfer with Unsupervised Source Reordering (N19-1)
Copied to clipboard
| Challenge: | Existing methods for dependency parsing use word order differences between source and target languages. |
| Approach: | They propose a cross-lingual transfer method that takes into account word order differences between source and target languages. |
| Outcome: | The proposed method improves on 68 treebanks (38 languages) on a target language. |
Cross-Lingual Dependency Parsing by POS-Guided Word Reordering (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to cross-lingual dependency parsing rely on large corpus size and cost. |
| Approach: | They propose a cross-lingual dependency parsing approach based on word reordering . they propose to train a model that transfers knowledge learned in one or multiple languages to target languages . |
| Outcome: | The proposed approach outperforms the baseline approach in Hindi and Latin by 15.3% and 6.7%. |
Word Reordering for Zero-shot Cross-lingual Structured Prediction (2021.emnlp-main)
Copied to clipboard
| Challenge: | Current sentence encoders are word order sensitive, resulting in poor performance . Adapting word order from one language to another is key in cross-lingual structured prediction. |
| Approach: | They propose a new module to organize words following the source language order . they build structured prediction models with bag-of-words inputs and introduce a module to do this . |
| Outcome: | The proposed model significantly improves target language performance for languages that are distant from the source language. |
A Simple and Effective Method to Improve Zero-Shot Cross-Lingual Transfer Learning (2022.coling-1)
Copied to clipboard
Kunbo Ding, Weijie Liu, Yuejian Fang, Weiquan Mao, Zhe Zhao, Tao Zhu, Haoyan Liu, Rong Tian, Yiren Chen
| Challenge: | Existing zero-shot cross-lingual transfer methods rely on parallel corpora or bilingual dictionaries . however, its effect is limited by the gap between embedding clusters of different languages . |
| Approach: | They propose Embedding-Push, Attention-Pull, and Robust targets to transfer English embeddings to virtual multilingual embedders without semantic loss. |
| Outcome: | Experimental results show that the proposed method outperforms existing methods on cross-lingual tasks and can achieve a better multilingual alignment. |
Understanding Cross-Lingual Alignment—A Survey (2024.findings-acl)
Copied to clipboard
| Challenge: | Cross-lingual alignment is the meaningful similarity of representations across languages in multilingual language models. |
| Approach: | They propose a taxonomy of methods to improve cross-lingual alignment . they argue that an effective trade-off between language-neutral and language-specific information is key . |
| Outcome: | The proposed methods can be applied to encoder models and encoder-decoder-only models . they show that language-neutral and language-specific information is key . |
Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsing (N19-1)
Copied to clipboard
| Challenge: | Existing methods for multilingual transfer are limited by their dynamic nature. |
| Approach: | They propose a method that utilizes deep contextual embeddings, pretrained in an unsupervised fashion. |
| Outcome: | The proposed method outperforms the state-of-the-art on 6 languages, yielding an improvement of 6.8 LAS points on average. |
Optimal Transport Posterior Alignment for Cross-lingual Semantic Parsing (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing work on cross-lingual semantic parsing has focused on English . a few-shot approach to parse from natural languages is comparatively unexplored . |
| Approach: | They propose a method that minimizes cross-lingual divergence between probabilistic latent variables by Optimal Transport. |
| Outcome: | The proposed method improves performance even without parallel input translations on two datasets. |
Make the Best of Cross-lingual Transfer: Evidence from POS Tagging with over 100 Languages (2022.acl-long)
Copied to clipboard
| Challenge: | Existing studies on cross-lingual generalisability of large pre-trained models use English training data and test data in unseen languages. |
| Approach: | They propose to use multilingual pre-trained models to model cross-lingual transfer in a selection of target languages. |
| Outcome: | The proposed model can be used to improve cross-lingual transfer performance in low-resource languages with no labeled training data. |
Disentangling Linguistic Relatedness from Task Alignment in Cross-Lingual Transfer (2026.acl-srw)
Copied to clipboard
| Challenge: | Large language models (LLMs) have advanced natural language processing, yet their benefits remain concentrated in English and a small number of high-resource languages. |
| Approach: | They fine-tuned large language models (4B–671B parameters) on Arabic and evaluated zero-shot reading comprehension on Semitic languages and non-Semitic controls. |
| Outcome: | The results show that models with weak baselines improve across all languages, whereas strong-baseline models show only marginal gains regardless of language family. |
Massively Multilingual Transfer for NER (P19-1)
Copied to clipboard
| Challenge: | Existing approaches for cross-lingual transfer use a single source language, but there are exceptions. |
| Approach: | They propose two techniques for modulating the transfer, suitable for zero-shot or few-shot learning, respectively. |
| Outcome: | The proposed methods are much more effective than baseline models and rival oracle selection of the single best individual model. |