MetaXL: Meta Representation Transformation for Low-resource Cross-lingual Learning (2021.naacl-main)
Copied to clipboard
Mengzhou Xia, Guoqing Zheng, Subhabrata Mukherjee, Milad Shokouhi, Graham Neubig, Ahmed Hassan Awadallah
| Challenge: | Recent work shows that multilingual representations are disjointed across languages, bringing additional challenges for transfer onto extremely low-resource languages. |
| Approach: | They propose a meta-learning based framework that learns to transform representations judiciously from auxiliary languages to a target one and brings their representation spaces closer for effective transfer. |
| Outcome: | The proposed framework learns to transform representations from auxiliary languages to a target language and brings their representation spaces closer for effective transfer. |
Similar Papers
Meta-XNLG: A Meta-Learning Approach Based on Language Clustering for Zero-Shot Cross-Lingual Transfer and Generation (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to learn shareable structures from low-resource languages are limited in the zero-shot setting. |
| Approach: | They propose a meta-learning framework to learn shareable structures from typologically diverse languages based on meta- learning and language clustering. |
| Outcome: | The proposed framework is able to learn shareable structures from typologically diverse languages with limited annotated data. |
T3L: Translate-and-Test Transfer Learning for Cross-Lingual Text Classification (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing approaches to cross-lingual text classification leverage text classifiers trained in a high-resource language to perform text classification in other languages with no or minimal fine-tuning. |
| Approach: | They propose to combine a neural machine translator and a text classifier trained in a high-resource language to perform text classification in other languages with no or minimal fine-tuning. |
| Outcome: | The proposed approach significantly improves over a baseline approach. |
Good Meta-tasks Make A Better Cross-lingual Meta-transfer Learning for Low-resource Languages (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Model-agnostic meta-learning has garnered attention as a promising technique for enhancing few-shot cross-lingual transfer learning in low-resource scenarios. |
| Approach: | They propose a Meta-Task Collector-based Cross-lingual Meta-Transfer framework to adapt data selection strategies to construct cross-lingual meta-tasks to reduce language gaps. |
| Outcome: | The proposed framework significantly improves model performance in the target language with minimal annotation costs. |
Meta-Learning for Low-Resource Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | In this paper, we propose to extend the recently introduced model-agnostic meta-learning algorithm for low-resource neural machine translation (NMT). |
| Approach: | They propose to extend the recently introduced meta-learning algorithm for low-resource neural machine translation (NMT) they frame low-Resource translation as a meta- learning problem where we learn to adapt to low-REsource languages based on multilingual high-resourced language tasks. |
| Outcome: | The proposed meta-learning algorithm outperforms the multilingual, transfer learning based approach and can train a competitive NMT system with only a fraction of training examples. |
Learn to Cross-lingual Transfer with Meta Graph Learning Across Heterogeneous Languages (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing mPLM-based methods focus on designing costly model pre-training while ignoring equally crucial downstream adaptation. |
| Approach: | They propose a meta graph learning method that extracts meta-knowledge from historical CLT experiences to learn to cross-lingual transfer. |
| Outcome: | The proposed method can learn to cross-lingual transfer by extracting meta-knowledge from historical CLT experiences (tasks) it can also capture intrinsic language relationships to explicitly guide cross-linguistic transfer. |
Multilingual and cross-lingual document classification: A meta-learning approach (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods to document classification in low-resource languages are under-resourced . 6% of the world's languages are spoken, and many have inadequate resources . |
| Approach: | They propose a meta-learning approach to document classification in low-resource languages . they propose 'nuclear-shot' cross-lingual adaptation to previously unseen languages based on limited data . |
| Outcome: | The proposed method performs on-par on some languages while under-resourced in others. |
Meta-Transfer Learning for Code-Switched Speech Recognition (2020.acl-main)
Copied to clipboard
| Challenge: | Increasing number of people in the world today speak a mixed-language as a result of being multilingual. |
| Approach: | They propose a method to transfer learn on a code-switched speech recognition system by extracting information from high-resource monolingual datasets. |
| Outcome: | The proposed model outperforms baselines on speech recognition and language modeling tasks and is faster to converge. |
Improving Low-Resource Languages in Pre-Trained Multilingual Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained multilingual language models are the foundation of many NLP approaches, but are often not well-supported by these models due to small available monolingual corpora. |
| Approach: | They propose an unsupervised approach to improve cross-lingual representations of low-resource languages by bootstrapping word translation pairs from monolingual corpora and using them to improve language alignment. |
| Outcome: | The proposed approach improves cross-lingual representations on low-resource languages using word retrieval and zero-shot named entity recognition. |
Meta Distant Transfer Learning for Pre-trained Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Notable PLMs are available for text classification tasks, but performance of PLM on downstream tasks may be limited by the availability of training set. |
| Approach: | They propose a meta-learning framework to learn the transferable knowledge across tasks using PLMs. |
| Outcome: | The proposed framework outperforms baselines on seven datasets and is task-agnostic and unbiased. |
Meta-Learning for Fast Cross-Lingual Adaptation in Dependency Parsing (2022.acl-long)
Copied to clipboard
Anna Langedijk, Verna Dankers, Phillip Lippe, Sander Bos, Bryan Cardenas Guevara, Helen Yannakoudakis, Ekaterina Shutova
| Challenge: | Meta-learning can help overcome resource scarcity in cross-lingual NLP problems . pre-training of models requires large annotated training sets for the task at hand . |
| Approach: | They propose to use meta-learning to train a model to learn a parameter initialization that can adapt quickly to new languages. |
| Outcome: | The proposed model-agnostic meta-learning improves on language transfer and standard supervised learning baselines for unseen, typologically diverse, and low-resource languages in a few-shot learning setup. |