A Recipe of Parallel Corpora Exploitation for Multilingual Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have highlighted the potential of exploiting parallel corpora to enhance multilingual large language models. |
| Approach: | They investigate the impact of parallel corpora quality and quantity, training objectives, and model size on performance of multilingual large language models enhanced with parallel corporeal. |
| Outcome: | The proposed approach improves performance in bilingual and general-purpose tasks. |
Similar Papers
BenchMAX: A Comprehensive Multilingual Evaluation Suite for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing multilingual benchmarks focus primarily on language understanding tasks. |
| Approach: | They develop a multi-way multilingual benchmark that measures critical capabilities of large language models across languages. |
| Outcome: | Extensive experiments on BenchMAX reveal uneven utilization of core capabilities across languages, emphasizing the performance gaps that scaling model size alone does not resolve. |
Is Translation All You Need? A Study on Solving Multilingual Tasks with Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated multilingual capabilities, yet they are mostly English-centric due to the imbalanced training corpora. |
| Approach: | They extend the evaluation to real-world user queries and non-English-centric LLMs . they show that translation into English can boost LLM performance on NLP tasks . |
| Outcome: | The proposed evaluation extends to user queries and non-English-centric LLMs . it shows that translation into English can boost performance on NLP tasks, but not universally optimal . |
An Analysis of Massively Multilingual Neural Machine Translation for Low-Resource Languages (2020.lrec-1)
Copied to clipboard
| Challenge: | In this study, we explore massively multilingual low-resource neural machine translation. |
| Approach: | They propose to use Bible translations to train models with up to 1,107 source languages and create multilingual corpora varying the number and relatedness of source languages. |
| Outcome: | The proposed approach is highly language-specific and can be tailored to the source language and its typology. |
A Multilingual Dataset for Evaluating Parallel Sentence Extraction from Comparable Corpora (L18-1)
Copied to clipboard
| Challenge: | BUCC Shared Task aims to extract parallel sentences from comparable corporad . resulting corpus contains about 3.5 million distinct sentences in english, french, german, Russian, and Chinese . |
| Approach: | They present challenges faced to build a parallel sentences dataset from comparable corporad . they emphasize issues faced to include Chinese as one of the languages . |
| Outcome: | The 2017 BUCC Shared Task was a first for this task . the dataset contains 3.5 million sentences in English, French, German, Russian, and Chinese . |
How Many Languages Make Good Multilingual Instruction Tuning? A Case Study on BLOOM (2025.coling-main)
Copied to clipboard
| Challenge: | Many large language models (LLMs) support many languages, while others only support a few, e.g. the Llama series. |
| Approach: | They present a case study on BLOOM to understand three pertinent factors affecting performance: the number of languages, language exposure, and similarity between training and test languages. |
| Outcome: | The proposed model can be used to perform multilingual tasks on 1 to 52 languages. |
A New Massive Multilingual Dataset for High-Performance Language Technologies (2024.lrec-main)
Copied to clipboard
Ona de Gibert, Graeme Nail, Nikolay Arefyev, Marta Bañón, Jelmer van der Linde, Shaoxiong Ji, Jaume Zaragoza-Bernabeu, Mikko Aulamo, Gema Ramírez-Sánchez, Andrey Kutuzov, Sampo Pyysalo, Stephan Oepen, Jörg Tiedemann
| Challenge: | a new massive multilingual dataset is available for language modeling and machine translation training. |
| Approach: | They present a massive multilingual dataset using web crawls from the Internet Archive and CommonCrawl . they use open-source software tools and high-performance computing to acquire, manage and process large corpora . |
| Outcome: | The HPLT language resources is a massive multilingual dataset . it includes monolingual and bilingual corpora extracted from CommonCrawl and the Internet Archive . the results are published online at the journal journal cense4 . |
Just Go Parallel: Improving the Multilingual Capabilities of Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have impressive translation capabilities even without being explicitly trained on parallel data. |
| Approach: | They propose to add parallel data to enhance multilingual encoder-based and encoder decoder language models by focusing on translation and multilingual common-sense reasoning. |
| Outcome: | The proposed methods show that adding parallel data can significantly improve LLMs’ multilingual capabilities. |
Language Lives in Sparse Dimensions: Toward Interpretable and Efficient Multilingual Control for Large Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | Prior studies show that large language models map multilingual content into English-aligned representations at intermediate layers before projecting them back into target-language token spaces in the later layers. |
| Approach: | They propose a method to identify and manipulate dimensions that are sparse and sparsity-based . they propose to use as few as 50 sentences of either parallel or monolingual data to manipulate these dimensions . |
| Outcome: | Experiments on a multilingual generation control task show the interpretability of these dimensions. |
The Role of Mixed-Language Documents for Multilingual Large Language Model Pretraining (2026.acl-long)
Copied to clipboard
Jiandong Shao, Raphael Tang, Crystina Zhang, Karin Sevegnani, Pontus Stenetorp, Jianfei Yang, Yao Lu
| Challenge: | Existing research suggests that multilingual large language models can achieve impressive cross-lingual understanding despite largely monolingual pretraining. |
| Approach: | They compare a monolingual-only corpus with a standard web corpus that removes all multilingual documents and then retrain the models from scratch under controlled conditions. |
| Outcome: | The results show that removing bilingual data causes translation performance to drop 56% in BLEU, whereas code-switching contributes minimally. |
nmT5 - Is parallel data still relevant for pre-training massively multilingual language models? (2021.acl-short)
Copied to clipboard
| Challenge: | Recent studies have shown that cross-lingual transfer learning in pre-trained multilingual models could be improved further by incorporating parallel data. |
| Approach: | They propose to integrate parallel data into mT5 pre-training to improve results on downstream multilingual and cross-lingual tasks. |
| Outcome: | The proposed model improves cross-lingual transfer significantly in small fine-tuning datasets and small model sizes. |