| Challenge: | In this paper, we introduce the Global Communication Plan (GCP) Corpus . the corpus is sentence-aligned and covers ten languages, including many Asian languages . |
| Approach: | They introduce the Global Communication Plan (GCP) Corpus, a multilingual parallel corpus . it is sentence-aligned and covers ten languages, including many Asian languages . |
| Outcome: | The proposed corpus is sentence-aligned and covers ten languages, including many Asian languages. |
Similar Papers
The DReaM Corpus: A Multilingual Annotated Corpus of Grammars for the World’s Languages (2020.lrec-1)
Copied to clipboard
| Challenge: | Until recently, language descriptions were available in paper form only, with indexes as the only search aid. |
| Approach: | They propose to digitize a multilingual corpus of language descriptions and annotate it with various meta, word, and text attributes to make searching and analysis easier and more useful. |
| Outcome: | The proposed corpus is searchable through a couple of well-established corpus infrastructures. |
A Multilingual Multiway Evaluation Data Set for Structured Document Translation of Asian Languages (2022.findings-aacl)
Copied to clipboard
| Challenge: | a lack of evaluation data sets for structured content limits progress in machine translation . a common use case of machine translation is the translation of structured or formatted documents . |
| Approach: | They propose a multilingual multiway evaluation data set for machine translation of structured documents of Asian languages Japanese, Korean and Chinese. |
| Outcome: | The proposed data set is well suited for multilingual evaluation and contains richer annotation tag sets than existing data sets. |
JW300: A Wide-Coverage Parallel Corpus for Low-Resource Languages (P19-1)
Copied to clipboard
| Challenge: | a shortage of parallel data in low-resource languages creates a bottleneck for cross-lingual transfer . a massive collection of parallel texts for over 300 diverse languages is our main contribution . |
| Approach: | They propose a parallel corpus of over 300 languages with 100 thousand parallel sentences per language pair on average. |
| Outcome: | The proposed dataset can be used to build cross-lingual word embeddings and multi-source part-of-speech projections. |
ParCourE: A Parallel Corpus Explorer for a Massively Multilingual Corpus (2021.acl-demo)
Copied to clipboard
| Challenge: | 7000 languages worldwide are spoken, but most research is focused on English . multilinguality is essential for multilingual research, and is a key component of the process. |
| Approach: | They propose a wordaligned parallel corpus that can be browsed using an online tool . they use the word alignment tools SimAlign and BabelNet to find the alignments . |
| Outcome: | The proposed tool can be set up for any parallel corpus and explores its quality and properties. |
A Corpus for Multilingual Document Classification in Eight Languages (L18-1)
Copied to clipboard
| Challenge: | a subset of the Reuters corpus volume 2 is used to evaluate cross-lingual document classification . current best practice is to evaluate document classification on resources in one language and transfer it to another without additional resources. |
| Approach: | They propose to use a subset of the Reuters corpus to evaluate cross-lingual document classification . they propose to add Italian, Russian, Japanese and Chinese to the subset . |
| Outcome: | The proposed subset of the Reuters corpus has balanced class priors for eight languages. |
KC4MT: A High-Quality Corpus for Multilingual Machine Translation (2022.lrec-1)
Copied to clipboard
Vinh Van Nguyen, Ha Nguyen, Huong Thanh Le, Thai Phuong Nguyen, Tan Van Bui, Luan Nghia Pham, Anh Tuan Phan, Cong Hoang-Minh Nguyen, Viet Hong Tran, Anh Huu Tran
| Challenge: | In machine translation, Vietnamese is a low-resource language, and the quality of the training corpus is very low. |
| Approach: | They propose a method for building high-quality multilingual parallel corpus in news domain . they also publicize a corpus that includes 500.000 Vietnamese-Chinese bilingual sentence pairs . |
| Outcome: | The proposed method improves the quality of multilingual machine translation in Vietnamese, Laos, and Khmer . the public version includes 500.000 Vietnamese-Chinese bilingual sentence pairs . |
From Unaligned to Aligned: Scaling Multilingual LLMs with Multi-Way Parallel Corpora (2025.emnlp-main)
Copied to clipboard
| Challenge: | Experiments show that models trained on multi-way parallel data outperform those trained on unaligned data. |
| Approach: | They propose a large-scale, high-quality multi-way parallel corpus based on TED Talks that spans 113 languages with up to 50 languages aligned in parallel. |
| Outcome: | The proposed model outperforms models trained on unaligned multilingual data on six multilingual benchmarks. |
A Multilingual Parallel Corpora Collection Effort for Indian Languages (2020.lrec-1)
Copied to clipboard
| Challenge: | Currently, neural network based approaches for machine translation are data hungry and sentence-level aligned parallel pairs are the currency. |
| Approach: | They propose to build sentence aligned parallel corpora across 10 Indian languages using online sources which have content shared across languages. |
| Outcome: | The proposed corpora significantly extends existing resources that are either not large enough or are restricted to a specific domain (such as health). |
XDailyDialog: A Multilingual Parallel Dialogue Corpus (2023.acl-long)
Copied to clipboard
Zeming Liu, Ping Nie, Jie Cai, Haifeng Wang, Zheng-Yu Niu, Peng Zhang, Mrinmaya Sachan, Kaiping Peng
| Challenge: | Existing datasets for open-domain dialogue modeling limited to a single language . absence of multilingual datasets hinders development of robust open- domain dialog systems . |
| Approach: | They propose a multilingual parallel open-domain dialog dataset to explore multilingual and cross-lingual open- domain dialog. |
| Outcome: | The proposed model can be used to explore multilingual and cross-lingual open-domain dialogs in other languages. |
A Multilingual Dataset for Evaluating Parallel Sentence Extraction from Comparable Corpora (L18-1)
Copied to clipboard
| Challenge: | BUCC Shared Task aims to extract parallel sentences from comparable corporad . resulting corpus contains about 3.5 million distinct sentences in english, french, german, Russian, and Chinese . |
| Approach: | They present challenges faced to build a parallel sentences dataset from comparable corporad . they emphasize issues faced to include Chinese as one of the languages . |
| Outcome: | The 2017 BUCC Shared Task was a first for this task . the dataset contains 3.5 million sentences in English, French, German, Russian, and Chinese . |