On the Relation between Syntactic Divergence and Zero-Shot Performance (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in cross-lingual transfer methods have enabled significant advances in grammatical processing tasks. |
| Approach: | They examine the extent to which syntactic relations are preserved in translation and parsability in a zero-shot setting. |
| Outcome: | The proposed model is based on a translation task in English and a subset of a standard English RE benchmark translated to Russian and Korean. |
Similar Papers
Frustratingly Simple but Surprisingly Strong: Using Language-Independent Features for Zero-shot Cross-lingual Semantic Parsing (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing training data is limited for languages other than English, so is the performance of the developed parsers. |
| Approach: | They propose to apply a pre-trained multilingual model to Italian, German and Dutch parsers where only a small number of manually annotated parses are available. |
| Outcome: | The proposed model improves on six parsers in English and Italian, German and Dutch, with the addition of universal dependency relations and universal POS tags as model-agnostic features. |
When is BERT Multilingual? Isolating Crucial Ingredients for Cross-lingual Transfer (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent work on multilingual language models has demonstrated their capacity for cross-lingual zero-shot transfer on downstream tasks. |
| Approach: | They conduct a large-scale empirical study to isolate the effects of various linguistic properties by measuring zero-shot transfer between four different natural languages. |
| Outcome: | The proposed model exhibits decent cross-lingual zero-shot transfer, with no significant differences in word order and embedding alignment. |
Zero-shot Dependency Parsing with Pre-trained Multilingual Sentence Representations (D19-61)
Copied to clipboard
| Challenge: | Pretrained sentence representations have set the new state of the art in many language understanding tasks. |
| Approach: | They propose to use a multilingual corpus to train deep bidirectional sentence representations that are fully lexicalized to allow for the development of an unsupervised universal dependency parser. |
| Outcome: | The proposed approach outperforms the best CoNLL 2018 systems in all of the shared task’s six truly low-resource languages while using a single system. |
On The Ingredients of an Effective Zero-shot Semantic Parser (2022.acl-long)
Copied to clipboard
| Challenge: | Recent studies have performed zero-shot learning by synthesizing training examples of canonical utterances and programs from a grammar, and further paraphrasing these utterrances to improve linguistic diversity. |
| Approach: | They propose to bridge gaps between canonical and real-world user-issued examples by using stronger paraphrasers and improved grammars. |
| Outcome: | The proposed model achieves strong performance on two semantic parsing benchmarks with zero labeled data. |
A Simple and Effective Method to Improve Zero-Shot Cross-Lingual Transfer Learning (2022.coling-1)
Copied to clipboard
Kunbo Ding, Weijie Liu, Yuejian Fang, Weiquan Mao, Zhe Zhao, Tao Zhu, Haoyan Liu, Rong Tian, Yiren Chen
| Challenge: | Existing zero-shot cross-lingual transfer methods rely on parallel corpora or bilingual dictionaries . however, its effect is limited by the gap between embedding clusters of different languages . |
| Approach: | They propose Embedding-Push, Attention-Pull, and Robust targets to transfer English embeddings to virtual multilingual embedders without semantic loss. |
| Outcome: | Experimental results show that the proposed method outperforms existing methods on cross-lingual tasks and can achieve a better multilingual alignment. |
Analyzing the Evaluation of Cross-Lingual Knowledge Transfer in Multilingual Language Models (2024.eacl-long)
Copied to clipboard
| Challenge: | Recent advances in training multilingual models on large datasets have shown promising results in knowledge transfer across languages. |
| Approach: | They challenge the assumption that high zero-shot performance reflects high cross-lingual ability by introducing more challenging setups involving instances with multiple languages. |
| Outcome: | The proposed model can achieve high performance on multilingual benchmarks and on low-resource languages. |
Narrowing the Gap between Zero- and Few-shot Machine Translation by Matching Styles (2024.findings-naacl)
Copied to clipboard
Weiting Tan, Haoran Xu, Lingfeng Shen, Shuyue Stella Li, Kenton Murray, Philipp Koehn, Benjamin Van Durme, Yunmo Chen
| Challenge: | Recent work shows that large language models can generalize to machine translation using zero-shot examples with in-context learning. |
| Approach: | They investigate the factors contributing to this gap by matching the writing styles of the target corpus. |
| Outcome: | The proposed methods can be enhanced without the need for parallel demonstration examples. |
Zero-Shot vs. Translation-Based Cross-Lingual Transfer: The Case of Lexical Gaps (2024.naacl-short)
Copied to clipboard
| Challenge: | lexical gaps exist in a variety of domains, such as QA, but they can only be expressed as a combination of words in another language. |
| Approach: | They compare the current performance and long-term viability of two approaches to cross-lingual transfer . they leverage lexical gaps to create a multilingual question answering dataset . |
| Outcome: | The proposed model outperforms zero-shot transfer and machine translation (MT) lexical gaps exist in a variety of domains, including linguistics, linguistic coding, and linguistic analysis. |
Fine-Grained Analysis of Cross-Linguistic Syntactic Divergences (2020.acl-main)
Copied to clipboard
Dmitry Nikolaev, Ofir Arviv, Taelin Karidi, Neta Kenneth, Veronika Mitnik, Lilja Maria Saeboe, Omri Abend
| Challenge: | Existing work on quantifying the prevalence of syntactic divergences across languages has not been done. |
| Approach: | They propose a framework for extracting divergence patterns for any language pair from a parallel corpus building on Universal Dependencies. |
| Outcome: | The proposed framework provides a detailed picture of cross-language divergences, generalizes previous approaches, and lends itself to full automation. |
Revisiting the Effects of Leakage on Dependency Parsing (2022.findings-acl)
Copied to clipboard
| Challenge: | Recent work shows that treebank size and linguistic variation are important factors that explain the variation in dependency parsing performance. |
| Approach: | They propose a measure of leakage that explains and correlates with observed performance variation. |
| Outcome: | The proposed measure explains and correlates with observed performance variation. |