Cross-Lingual Transfer in Zero-Shot Cross-Language Entity Linking (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing work on cross-language entity linking grounds mentions written in multiple languages to a monolingual knowledge base is lacking. |
| Approach: | They propose a task that uses multilingual BERT representations of both the mention and context as input and explore zero-shot language transfer. |
| Outcome: | The proposed model performs well in both monolingual and multilingual settings. |
Similar Papers
A Simple and Effective Method to Improve Zero-Shot Cross-Lingual Transfer Learning (2022.coling-1)
Copied to clipboard
Kunbo Ding, Weijie Liu, Yuejian Fang, Weiquan Mao, Zhe Zhao, Tao Zhu, Haoyan Liu, Rong Tian, Yiren Chen
| Challenge: | Existing zero-shot cross-lingual transfer methods rely on parallel corpora or bilingual dictionaries . however, its effect is limited by the gap between embedding clusters of different languages . |
| Approach: | They propose Embedding-Push, Attention-Pull, and Robust targets to transfer English embeddings to virtual multilingual embedders without semantic loss. |
| Outcome: | Experimental results show that the proposed method outperforms existing methods on cross-lingual tasks and can achieve a better multilingual alignment. |
Cross-Lingual BERT Transformation for Zero-Shot Dependency Parsing (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to learn cross-lingual word embeddings in a contextual space are lacking. |
| Approach: | They propose a method to generate cross-lingual contextualized word embeddings using pre-trained BERT models by learning a linear transformation from contextual word alignments. |
| Outcome: | The proposed approach outperforms state-of-the-art models on zero-shot cross-lingual transfer parsing and is highly competitive with existing models. |
Can Monolingual Pretrained Models Help Cross-Lingual Classification? (2020.aacl-main)
Copied to clipboard
| Challenge: | Multilingual pretrained language models have shown impressive results for cross-lingual transfer, but due to the constant model capacity, multilingual pre-training usually lags behind the monolingual competitors. |
| Approach: | They propose to transfer the knowledge from monolingual pretrained models to multilingual ones to improve zero-shot cross-lingual classification by using machine translation systems. |
| Outcome: | The proposed methods outperform vanilla multilingual fine-tuning on two cross-lingual classification benchmarks. |
Joint Multilingual Supervision for Cross-lingual Entity Linking (D18-1)
Copied to clipboard
| Challenge: | Entity Linking (XEL) systems ground entity mentions written in any language to Wikipedia . XEL is challenging for most languages due to limited availability of resources as supervision . |
| Approach: | They develop a cross-lingual XEL approach that combines supervision from multiple languages jointly. |
| Outcome: | The proposed approach significantly improves on the current state-of-the-art in 8 languages. |
Zero-shot Dependency Parsing with Pre-trained Multilingual Sentence Representations (D19-61)
Copied to clipboard
| Challenge: | Pretrained sentence representations have set the new state of the art in many language understanding tasks. |
| Approach: | They propose to use a multilingual corpus to train deep bidirectional sentence representations that are fully lexicalized to allow for the development of an unsupervised universal dependency parser. |
| Outcome: | The proposed approach outperforms the best CoNLL 2018 systems in all of the shared task’s six truly low-resource languages while using a single system. |
Don’t Use English Dev: On the Zero-Shot Cross-Lingual Evaluation of Contextual Embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual contextual embeddings have demonstrated state-of-the-art performance in zero-shot cross-lingual transfer learning. |
| Approach: | They show that English dev accuracy makes it difficult to obtain reproducible results . they recommend providing oracle scores alongside zero-shot results if possible . |
| Outcome: | mBERT and XLM have shown strong performance on cross-lingual recognition, text classification, dependency parsing, and other tasks. |
Scalable Zero-shot Entity Linking with Dense Entity Retrieval (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for entity linking use manually curated mention tables and incoming Wikipedia link popularity. |
| Approach: | They propose a BERT-based entity linking model with a bi-encoder that embeds the mention context and the entity descriptions and then re-ranked the candidate with . they also evaluate the accuracy-speed trade-off inherent to large pre-trained models. |
| Outcome: | The proposed model is state-of-the-art on recent zero-shot benchmarks and established non-zero-shot evaluations. |
Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT (D19-1)
Copied to clipboard
| Challenge: | Pretrained contextual representation models have pushed forward the state-of-the-art on many NLP tasks. |
| Approach: | They propose to use a model that is pretrained on 104 languages for cross-lingual transfer. |
| Outcome: | The proposed model performs well on 5 NLP tasks covering 39 languages from various language families. |
Zero-shot Cross-lingual NER via Mitigating Language Difference: An Entity-aligned Translation Perspective (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to cross-lingual Named Entity Recognition focus on Latin script language (LSL) for non-Latin script language, performance often degrades due to deep structural differences. |
| Approach: | They propose an entity-aligned translation approach to align entities between NSL and English . |
| Outcome: | The proposed approach aims to transfer knowledge from high-resource languages to low-resourced languages. |
When is BERT Multilingual? Isolating Crucial Ingredients for Cross-lingual Transfer (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent work on multilingual language models has demonstrated their capacity for cross-lingual zero-shot transfer on downstream tasks. |
| Approach: | They conduct a large-scale empirical study to isolate the effects of various linguistic properties by measuring zero-shot transfer between four different natural languages. |
| Outcome: | The proposed model exhibits decent cross-lingual zero-shot transfer, with no significant differences in word order and embedding alignment. |