Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsing (N19-1)
Copied to clipboard
| Challenge: | Existing methods for multilingual transfer are limited by their dynamic nature. |
| Approach: | They propose a method that utilizes deep contextual embeddings, pretrained in an unsupervised fashion. |
| Outcome: | The proposed method outperforms the state-of-the-art on 6 languages, yielding an improvement of 6.8 LAS points on average. |
Similar Papers
A Simple and Effective Method to Improve Zero-Shot Cross-Lingual Transfer Learning (2022.coling-1)
Copied to clipboard
Kunbo Ding, Weijie Liu, Yuejian Fang, Weiquan Mao, Zhe Zhao, Tao Zhu, Haoyan Liu, Rong Tian, Yiren Chen
| Challenge: | Existing zero-shot cross-lingual transfer methods rely on parallel corpora or bilingual dictionaries . however, its effect is limited by the gap between embedding clusters of different languages . |
| Approach: | They propose Embedding-Push, Attention-Pull, and Robust targets to transfer English embeddings to virtual multilingual embedders without semantic loss. |
| Outcome: | Experimental results show that the proposed method outperforms existing methods on cross-lingual tasks and can achieve a better multilingual alignment. |
Cross-Lingual BERT Transformation for Zero-Shot Dependency Parsing (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to learn cross-lingual word embeddings in a contextual space are lacking. |
| Approach: | They propose a method to generate cross-lingual contextualized word embeddings using pre-trained BERT models by learning a linear transformation from contextual word alignments. |
| Outcome: | The proposed approach outperforms state-of-the-art models on zero-shot cross-lingual transfer parsing and is highly competitive with existing models. |
A Simple Approach to Learning Unsupervised Multilingual Embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on unsupervised cross-lingual embeddings in the bilingual setting has given the impetus to learning a shared embeddable space for several languages. |
| Approach: | They propose to solve two sub-problems together to learn a shared embedding space for several languages. |
| Outcome: | The proposed approach outperforms existing methods in bilingual lexicon induction, cross-lingual word similarity, multilingual document classification, and multilingual dependency parsing tasks. |
Beyond Offline Mapping: Learning Cross-lingual Word Embeddings through Context Anchoring (2021.acl-long)
Copied to clipboard
| Challenge: | Recent research on cross-lingual word embeddings has been dominated by unsupervised mapping approaches that align monolingual embedders. |
| Approach: | They propose an unsupervised mapping approach that fixes fixed embeddings and learns new ones for the source language that are aligned with them. |
| Outcome: | The proposed method outperforms conventional mapping methods on bilingual lexicon induction and obtains competitive results in the downstream XNLI task. |
Zero-shot Dependency Parsing with Pre-trained Multilingual Sentence Representations (D19-61)
Copied to clipboard
| Challenge: | Pretrained sentence representations have set the new state of the art in many language understanding tasks. |
| Approach: | They propose to use a multilingual corpus to train deep bidirectional sentence representations that are fully lexicalized to allow for the development of an unsupervised universal dependency parser. |
| Outcome: | The proposed approach outperforms the best CoNLL 2018 systems in all of the shared task’s six truly low-resource languages while using a single system. |
Improving Cross-lingual Transfer through Subtree-aware Word Reordering (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that multilingual language models are not effective when dealing with less-represented languages. |
| Approach: | They propose a powerful reordering method that learns word-order patterns conditioned on the syntactic context from a small amount of annotated data. |
| Outcome: | The proposed method outperforms baselines on a variety of tasks and is effective in both zero-shot and few-shot scenarios. |
Examining Cross-lingual Contextual Embeddings with Orthogonal Structural Probes (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on whether multilingual embeddings can be aligned in a shared space across languages are lacking. |
| Approach: | They propose to learn a projection based on monolingual annotated datasets and evaluate syntactic and lexical information encoded in a shared cross-lingual embedding space. |
| Outcome: | The proposed model can be used to learn representations for languages with low resources. |
Optimal Transport Posterior Alignment for Cross-lingual Semantic Parsing (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing work on cross-lingual semantic parsing has focused on English . a few-shot approach to parse from natural languages is comparatively unexplored . |
| Approach: | They propose a method that minimizes cross-lingual divergence between probabilistic latent variables by Optimal Transport. |
| Outcome: | The proposed method improves performance even without parallel input translations on two datasets. |
Cross-Align: Modeling Deep Cross-lingual Interactions for Word Alignment (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing word alignment models capture few interactions between input sentence pairs, which severely degrades the word alignment quality. |
| Approach: | They propose to model deep interactions between input and target sentences using a two-stage training framework to train the model. |
| Outcome: | The proposed model achieves the state-of-the-art (SOTA) performance on four out of five language pairs. |
Improving Cross-Lingual Word Embeddings by Meeting in the Middle (D18-1)
Copied to clipboard
| Challenge: | Cross-lingual word embeddings are becoming increasingly important in multilingual NLP. |
| Approach: | They propose to apply an additional transformation after initial alignment to align two disjoint monolingual vector spaces. |
| Outcome: | The proposed approach outperforms state-of-the-art models in monolingual and cross-lingual evaluation tasks. |