Papers with XQuAD
How to Translate Your Samples and Choose Your Shots? Analyzing Translate-train & Few-shot Cross-lingual Transfer (2022.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have focused on zero-shot cross-lingual transfer of pretrained languages. |
| Approach: | They propose to use few-shot cross-lingual transfer to improve zero-shot performance of multilingual pretrained language models. |
| Outcome: | The proposed model can be scaled to high-quality samples and improves on zero-shot performance. |
Learning Disentangled Semantic Representations for Zero-Shot Cross-Lingual Transfer in Multilingual Machine Reading Comprehension (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods to zero-shot transfer knowledge from rich-resource to low-resourced languages are limited due to linguistic discrepancies in different languages. |
| Approach: | They propose a multilingual MRC framework equipped with a Siamese Semantic Disentanglement Model to disassociate semantics from syntax in models learned by multilingual pre-trained models. |
| Outcome: | The proposed model disassociates semantics from syntax in multilingual models. |
Multi-level Distillation of Semantic Knowledge for Pre-training Multilingual Language Model (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for improving multilingual models did not focus on learning the semantic structure of representation. |
| Approach: | They propose a method to improve multilingual language models by aligning parallel sentences . they propose token-, word-, sentence- and structure-level alignment objectives . |
| Outcome: | The proposed method outperforms baseline models on XNLI, PAWS-X, and XQuAD . it obtains comparable performance on low-resource languages, the authors show . |
A Measure for Transparent Comparison of Linguistic Diversity in Multilingual NLP Data Sets (2024.findings-naacl)
Copied to clipboard
| Challenge: | a new study aims to assess linguistic diversity of multilingual data sets against a reference language sample . linguistic diversity is typically measured as the number of languages included in the data set . but such measures do not consider structural properties of the included languages . |
| Approach: | They propose to measure linguistic diversity against a reference language sample to maximise linguistic diversity. |
| Outcome: | The proposed measure can be used to identify the types of languages that are not represented in a data set. |
Using Optimal Transport as Alignment Objective for fine-tuning Multilingual Contextualized Embeddings (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies suggest different methods to improve multilingual word representations in contextualized settings including techniques that align between source and target embedding spaces. |
| Approach: | They propose to use Optimal Transport as an alignment objective during fine-tuning to improve multilingual contextualized representations for downstream cross-lingual transfer. |
| Outcome: | The proposed method achieves better performance on two tasks (XNLI and XQuAD) and is competitive with existing methods. |
Investigating Transfer Learning in Multilingual Pre-trained Language Models through Chinese Natural Language Inference (2021.findings-acl)
Copied to clipboard
| Challenge: | Multilingual transformers have been shown to have remarkable transfer skills in zero-shot settings. |
| Approach: | They investigate cross-lingual transfer abilities of XLM-R for Chinese and English natural language inference using a large scale Chinese dataset. |
| Outcome: | The proposed model trains on Chinese and English natural language inference datasets. |
On the Cross-lingual Transferability of Monolingual Representations (2020.acl-main)
Copied to clipboard
| Challenge: | State-of-the-art unsupervised multilingual models generalize in zero-shot cross-lingual setting . generalization ability attributed to shared subword vocabulary and joint training across multiple languages . |
| Approach: | They propose an approach that transfers a monolingual model to new languages at the lexical level. |
| Outcome: | The proposed approach is competitive with multilingual BERT on cross-lingual classification benchmarks and on a new cross-linguistic question answering dataset. |
Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answering (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to improve Question Answering performance on non-English data are expensive and limited to evaluation set. |
| Approach: | They propose a method to improve Question Answering performance without additional annotations by leveraging Question Generation models to produce synthetic samples in a cross-lingual fashion. |
| Outcome: | The proposed method outperforms baselines on four datasets in English significantly . the proposed model outperformed baselines in english and is comparable to the validation set of the original SQuAD. |
Automatic Spanish Translation of SQuAD Dataset for Multi-lingual Question Answering (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods to train multilingual QA systems are limited for other languages . cross-lingual learning is a technique that transfers knowledge from source to target language with fewer training data. |
| Approach: | They propose a translation method to translate the Stanford Question Answering Dataset to Spanish and a multilingual-BERT model to train Spanish QA systems. |
| Outcome: | The proposed method outperforms the previous benchmarks for cross-lingual extractive QA. |
XLM-V: Overcoming the Vocabulary Bottleneck in Multilingual Masked Language Models (2023.emnlp-main)
Copied to clipboard
Davis Liang, Hila Gonen, Yuning Mao, Rui Hou, Naman Goyal, Marjan Ghazvininejad, Luke Zettlemoyer, Madian Khabsa
| Challenge: | Large multilingual models rely on a single vocabulary shared across 100+ languages . this vocabulary bottleneck limits the representational capabilities of multilingual model XLM-R . |
| Approach: | They propose a new approach for scaling to large multilingual vocabularies by de-emphasizing token sharing between languages with little lexical overlap and assigning vocabulary capacity to achieve sufficient coverage for each individual language. |
| Outcome: | The proposed model outperforms XLM-R on all language tasks and is particularly effective on low-resource tasks. |
False Friends Are Not Foes: Investigating Vocabulary Overlap in Multilingual Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work has shown that token overlap facilitates cross-lingual transfer or introduces interference between languages? |
| Approach: | They devised a controlled experiment where they train bilingual autoregressive models on multiple language pairs under systematically varied vocabulary overlap settings. |
| Outcome: | The proposed model outperforms models with disjointed vocabularies on XNLI and XQuAD and shows that token overlap is beneficial for multilingual tokenizers. |