Translate to Disambiguate: Zero-shot Multilingual Word Sense Disambiguation with Pretrained Language Models (2024.eacl-long)
Copied to clipboard
| Challenge: | Pretrained language models learn cross-lingual knowledge and perform well on diverse tasks when finetuned. |
| Approach: | They propose a zero-shot prompting approach that captures cross-lingual word sense with a contextual prompt. |
| Outcome: | The proposed approach outperforms baselines on recall in many evaluation languages without additional training or finetuning. |
Similar Papers
Combating the Curse of Multilinguality in Cross-Lingual WSD by Aligning Sparse Contextualized Word Representations (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to handle knowledge acquisition bottlenecks in multilingual training are limited due to the curse of multilinguality. |
| Approach: | They propose to use large pre-trained monolingual language models in cross lingual zero-shot word sense disambiguation coupled with a contextualized mapping mechanism. |
| Outcome: | The proposed model improves the average F-score by nearly 6.5 points over 17 target languages. |
Don’t Use English Dev: On the Zero-Shot Cross-Lingual Evaluation of Contextual Embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual contextual embeddings have demonstrated state-of-the-art performance in zero-shot cross-lingual transfer learning. |
| Approach: | They show that English dev accuracy makes it difficult to obtain reproducible results . they recommend providing oracle scores alongside zero-shot results if possible . |
| Outcome: | mBERT and XLM have shown strong performance on cross-lingual recognition, text classification, dependency parsing, and other tasks. |
Pre-trained Language Models Can be Fully Zero-Shot Learners (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to pre-trained language models require fine-tuning on labeled datasets or manually constructing proper prompts. |
| Approach: | They propose a nonparametric prompting PLM for fully zero-shot language understanding . they compare it to previous methods for text classification and text entailment . |
| Outcome: | The proposed method outperforms previous methods on diverse tasks. |
Zero-shot Cross-lingual Transfer of Prompt-based Tuning with a Unified Multilingual Prompt (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing work focuses on monolingual prompts, but we study multilingual prompt for multilingual models. |
| Approach: | They propose a model that uses a unified prompt for all languages, called UniPrompt, to alleviate the effort of designing different prompts for multiple languages. |
| Outcome: | The proposed model outperforms baseline models in the zero-shot cross-lingual setting. |
Avoiding Inference Heuristics in Few-shot Prompt-based Finetuning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent prompt-based approaches allow pretrained language models to achieve strong performances on few-shot finetuning by reformulating downstream task instances as a language modeling problem. |
| Approach: | They propose to reformulate downstream tasks as a language modeling problem and add a regularization that preserves pretraining weights to the model to mitigate the destructive tendency of few-shot finetuning. |
| Outcome: | The proposed model performs better on low data regimes than the standard model on few-shot finetuning. |
Cross-Lingual BERT Transformation for Zero-Shot Dependency Parsing (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to learn cross-lingual word embeddings in a contextual space are lacking. |
| Approach: | They propose a method to generate cross-lingual contextualized word embeddings using pre-trained BERT models by learning a linear transformation from contextual word alignments. |
| Outcome: | The proposed approach outperforms state-of-the-art models on zero-shot cross-lingual transfer parsing and is highly competitive with existing models. |
Discrete and Soft Prompting for Multilingual Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | In few-shot learning, discrete and soft prompting perform better than finetuning in multilingual cases. |
| Approach: | They show that discrete and soft prompting perform better than finetuning in crosslingual transfer and in-language training of multilingual natural language inference. |
| Outcome: | The proposed prompting model outperforms finetuning in crosslingual transfer and in-language training of multilingual natural language inference. |
Cross-Lingual Transfer in Zero-Shot Cross-Language Entity Linking (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing work on cross-language entity linking grounds mentions written in multiple languages to a monolingual knowledge base is lacking. |
| Approach: | They propose a task that uses multilingual BERT representations of both the mention and context as input and explore zero-shot language transfer. |
| Outcome: | The proposed model performs well in both monolingual and multilingual settings. |
A Simple and Effective Method to Improve Zero-Shot Cross-Lingual Transfer Learning (2022.coling-1)
Copied to clipboard
Kunbo Ding, Weijie Liu, Yuejian Fang, Weiquan Mao, Zhe Zhao, Tao Zhu, Haoyan Liu, Rong Tian, Yiren Chen
| Challenge: | Existing zero-shot cross-lingual transfer methods rely on parallel corpora or bilingual dictionaries . however, its effect is limited by the gap between embedding clusters of different languages . |
| Approach: | They propose Embedding-Push, Attention-Pull, and Robust targets to transfer English embeddings to virtual multilingual embedders without semantic loss. |
| Outcome: | Experimental results show that the proposed method outperforms existing methods on cross-lingual tasks and can achieve a better multilingual alignment. |
FEWS: Large-Scale, Low-Shot Word Sense Disambiguation with the Dictionary (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing models for Word Sense Disambiguation struggle to disambiguate rare senses . current models struggle to learn senses with few training examples . |
| Approach: | They introduce a low-shot WSD dataset automatically extracted from example sentences in Wiktionary. |
| Outcome: | The proposed dataset outperforms baseline models on rare senses in existing datasets. |