Zero-shot Learning for Grapheme to Phoneme Conversion with Language Ensemble (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing work focuses on low-resource and endangered languages with limited training sets. |
| Approach: | They propose a hypothesis set for any unseen target language and combine it with a confusion network to propose 'the most likely hypothesis' they test the approach on over 600 unseened languages and demonstrate it significantly outperforms baselines. |
| Outcome: | The proposed model outperforms baselines on over 600 unseen languages. |
Similar Papers
Multimodal, Multilingual Grapheme-to-Phoneme Conversion for Low-Resource Languages (D19-61)
Copied to clipboard
| Challenge: | Grapheme-to-phoneme conversion (g2p) is a task of predicting the pronunciation of words from their orthographic representation. |
| Approach: | They propose to leverage audio data as an auxiliary modality in a multi-task training process to learn a more optimal grapheme representation. |
| Outcome: | The proposed model reduces phoneme error rate to 2.46% on in-domain test set compared to unimodal spelling- pronunciation model. |
A Simple and Effective Method to Improve Zero-Shot Cross-Lingual Transfer Learning (2022.coling-1)
Copied to clipboard
Kunbo Ding, Weijie Liu, Yuejian Fang, Weiquan Mao, Zhe Zhao, Tao Zhu, Haoyan Liu, Rong Tian, Yiren Chen
| Challenge: | Existing zero-shot cross-lingual transfer methods rely on parallel corpora or bilingual dictionaries . however, its effect is limited by the gap between embedding clusters of different languages . |
| Approach: | They propose Embedding-Push, Attention-Pull, and Robust targets to transfer English embeddings to virtual multilingual embedders without semantic loss. |
| Outcome: | Experimental results show that the proposed method outperforms existing methods on cross-lingual tasks and can achieve a better multilingual alignment. |
ZGUL: Zero-shot Generalization to Unseen Languages using Multi-source Ensembling of Language Adapters (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to zero-shot cross-lingual transfer have focused on training with adapters of a single source and testing either with the target LA or LA of another related language. |
| Approach: | They propose to leverage LAs of multiple (linguistically or geographically related) source languages for more effective cross-lingual transfer instead of just one source LA . they extend their novel neural architecture, ZGUL, to settings where either (1) some unlabeled data or (2) few-shot training examples are available for the target language . |
| Outcome: | Extensive experimentation across four language groups, covering 15 unseen target languages, shows improvements of up to 3.2 average F1 points over baselines on POS tagging and NER tasks. |
Zero- and Few-Shot NLP with Pretrained Language Models (2022.acl-tutorials)
Copied to clipboard
| Challenge: | a tutorial aims to introduce NLP researchers to the latest techniques for learning from little-to-no data . aims at bringing interested researchers up to speed about the latest and ongoing techniques . |
| Approach: | They aim to introduce techniques for learning from little-to-no data using pretrained language models. |
| Outcome: | This tutorial aims to bring interested NLP researchers up to speed about recent techniques . it will cover methods from manual engineering, better inference algorithms to better tuning methods . |
Few-shot Learning with Multilingual Generative Language Models (2022.emnlp-main)
Copied to clipboard
Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O’Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, Mona Diab, Veselin Stoyanov, Xian Li
| Challenge: | Large-scale generative language models such as GPT-3 are competitive few-shot learners. |
| Approach: | They train multilingual generative language models on a corpus covering a diverse set of languages and study their few- and zero-shot learning capabilities. |
| Outcome: | The proposed model outperforms GPT-3 on 171 out of 182 directions with 32 training examples and surpasses the official supervised baseline in 45 directions. |
Zero-Shot Cross-Lingual NER Using Phonemic Representations for Low-Resource Languages (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing zero-shot cross-lingual NER approaches require substantial prior knowledge of the target language, which is impractical for low-resource languages. |
| Approach: | They propose a phonemic representation based on the International Phonetic Alphabet (IPA) to bridge the gap between representations of different languages. |
| Outcome: | The proposed method outperforms baseline models in low-resource languages with highest average F1 score and lowest standard deviation. |
Zero-shot Dependency Parsing with Pre-trained Multilingual Sentence Representations (D19-61)
Copied to clipboard
| Challenge: | Pretrained sentence representations have set the new state of the art in many language understanding tasks. |
| Approach: | They propose to use a multilingual corpus to train deep bidirectional sentence representations that are fully lexicalized to allow for the development of an unsupervised universal dependency parser. |
| Outcome: | The proposed approach outperforms the best CoNLL 2018 systems in all of the shared task’s six truly low-resource languages while using a single system. |
Multi Task Learning For Zero Shot Performance Prediction of Multilingual Models (2022.acl-long)
Copied to clipboard
| Challenge: | Massively Multilingual Transformer based Language Models have been shown to be effective on zero-shot transfer across languages, though performance varies from language to language depending on pivot language(s) used for fine-tuning. |
| Approach: | They propose to combine multi-task learning problems with multi-lingual Transformers to model zero-shot transfer across languages. |
| Outcome: | The proposed model can predict zero-shot transfer across languages with a multi-task learning problem with pretraining data in very few languages. |
Zero-shot Reading Comprehension by Cross-lingual Transfer Learning with Multi-lingual Language Representation Model (D19-1)
Copied to clipboard
| Challenge: | Existing studies on RC datasets in English have limited results due to lack of training data. |
| Approach: | They systematically explore zero-shot cross-lingual transfer learning on reading comprehension tasks with pre-trained language representation model. |
| Outcome: | The proposed model performs well on reading comprehension tasks on pre-trained language representation models. |
Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | a new study examines zero-shot cross-lingual transfer of vision-language models . we study multilingual text-to-video search in non-English languages without annotations . |
| Approach: | They propose a Transformer-based model that learns contextual multilingual multimodal embeddings . they propose 'zero-shot cross-lingual transfer' to improve multilingual search . |
| Outcome: | The proposed model outperforms baselines on multilingual text-to-video search and multilingual image search on VTT and VATEX. |